CITI Program is not unique in its mortifying stupidity

Writing in Slate, L. V. Anderson condemns simplistic, online training programs that are supposed to encourage regulatory compliance, but really just suck up time and money without improving behavior.

[L. V. Anderson, “Ethics Trainings Are Even Dumber Than You Think,” Slate, May 19, 2016.]

Anderson writes,

Regulators, managers, and employees are caught in a vicious cycle. Regulators pressure companies to implement training programs in hopes of reducing corporate crime and malfeasance. Executives implement training programs in hopes of protecting themselves against lawsuits and prosecution. Employees see through executives’ motivations and ignore, or even rebel against, the lessons of the trainings.

Although there’s not much research one way or the other, the online nature of compliance courses probably exacerbates this vicious cycle.

Anderson does not specifically mention the mortifyingly stupid CITI Program and its cousins in the IRB world, but everything she says applies to them.

Gentle Regulation May Be More Effective

Law professor Samuel Bagenstos argues that recent Title IX excesses follow the pattern of IRB horror stories: the feds threaten drastic action, so university administrators hyper-regulate. He offers disability rights as an example of a less punitive regulatory effort that has produced good results.

[Samuel R. Bagenstos, “What Went Wrong With Title IX?,” Washington Monthly, October 2015.]

On Signing the Markingson Petition

By April 1942, the Pentagon was 40 percent over budget, partly because it had been enlarged since first approved, but mostly because the original estimate of $35 million had never been realistic. Lieutenant General Brehon Somervell delayed telling Congress, but in June he finally sent Colonel Leslie Groves to appear before a House Apppropriations subcommittee.

Dunn and Hunter Defend Mandatory Review

The Journal of Medical Ethics has published two responses to Murray Dyck and Gary Allen's August 2012 article, “Is Mandatory Research Ethics Reviewing Ethical?” The responses do little to grapple with what I take to be the article's major's proposal.

[Michael Dunn, “Getting the Justification for Research Ethics Review Right.” Journal of Medical Ethics (October 31, 2012). doi:10.1136/medethics-2012-100943; David Hunter, “How Not to Argue Against Mandatory Ethics Review.” Journal of Medical Ethics (December 12, 2012). doi:10.1136/medethics-2012-101074.]

Could Guidance and Feedback Replace Rote Compliance?

Murray Dyck and Gary Allen, both of Griffith University in Australia, argue that "the review process should be an advisory and collegial one—not one that focuses on compliance, enforcement and gatekeeping."

[Murray Dyck and Gary Allen. “Is Mandatory Research Ethics Reviewing Ethical?” Journal of Medical Ethics (August 3, 2012), DOI: 10.1136/medethics-2011-100274.]

Can an IRB Ban a Researcher?

In July, the Sacramento Bee reported that the University of California, Davis, had ordered two doctors--J. Paul Muizelaar and Dr. Rudolph J. Schrot--"to halt all human research activity 'except as necessary to protect the safety and welfare of research participants.'" Schrot told the Bee that "To be banned from clinical research makes a career in academic medicine challenging, to say the least."

The Common Rule (45 CFR 46.113) empowers IRBs to "suspend or terminate approval of research that is not being conducted in accordance with the IRB's requirements or that has been associated with unexpected serious harm to subjects," but it does not explicitly empower them to ban a researcher from all research.

IRB Sought to Monitor Interviews with Elected Officials

The first article in the Professional Geographer special issue argues that the IRB system assumes that the researcher is a "powerful, knowing agent who assembles a scientific methodology that is always of potential harm to the researched," while subjects are always "less knowing" and vulnerable. As a result, scholars "face the presumption of guilt while seeking to prove innocence in the IRB process of application, negotiation, and usually, but not painlessly, final approval to conduct research."

[Deborah G. Martin and Joshua Inwood. “Subjectivity, Power, and the IRB,” Professional Geographer 64, no. 1 (2012): 7–15, DOI:10.1080/00330124.2011.596781.]

Harvard Law Today Reports on ANPRM Conference

Harvard Law Today, published by Harvard Law School, reports on May's conference on the ANPRM, held at the school's Petrie-Flom Center for Health Law Policy, Biotechnology, and Bioethics.

["The Future of Human Subjects Research Regulation," Harvard Law Today, July 2012.]

The article highlights the plenary address by Greg Koski, former director of OHRP.

Koski said regulation has been dominated by human-subject protectionism and an ethical-review system that has “devolved to regulatory compliance oversight.” The result, he said, is a system that is “inefficient and burdensome.” His recommendation, he said, was to replace the current system with one modeled after medical training and certification.

He said: “If we were able to develop a paradigm of professionalism in human research, it would likely be every bit as effective, less costly, less burdensome and more efficient than the protectionist, compliance-focused system that we are now seeking to reform. I would argue that reform of our current system is perhaps not the most appropriate or even adequate approach to try to achieve the goals that we seek.”

Defino Notes Big Drop in OHRP Letters and Cases

Theresa Defino, editor of Report on Research Compliance, kindly alerted me to her recent analysis of OHRP activity and allowed me to post a reprint of the article.

["Big Drop in OHRP Letters, Open Cases Raise Questions of Agency Commitment," Report on Research Compliance, March 2011, 1-3.]

Menikoff Passes the Buck

Joseph Millum, bioethicist at the National Institutes of Health, and Jerry Menikoff, director of the Office for Human Research Protections, acknowledge the widespread dissatisfaction with present human subjects regulations and wish that "ethics review could be streamlined under the current regulations if institutions, IRBs, and researchers adhered strictly to the definition of human subjects research and used the available options for exemptions, expedited review, and centralized review—options that remain underused in biomedical research." But they put too much blame for this overregulation on IRBs and research institutions rather than on their own agencies.

[Joseph Millum and Jerry Menikoff, "Streamlining Ethical Review," Annals of Internal Medicine 153, no. 10 (November 15, 2010): 655-657.]

Twenty-Six Percent of Boxes Go Unchecked

Approximately 74 percent of U.S. institutions with federalwide assurances apply all or part of 45 CFR 46 to all research regardless of support, down from more than 90 percent of such institutions a decade or so ago, according to OHRP officials.

[Carol Weil, Lisa Rooney, Patrick McNeilly, Karena Cooper, Kristina Borror, and Paul Andreason, "OHRP Compliance Oversight Letters: An Update," IRB: Ethics & Human Research 32, no. 2 (2010): 1-6.]

The authors write that since their previous review of determination letters, published in 2003 and covering the period 1998-2002,

more institutions have decided not to extend their FWA to research not supported by HHS, which means that OHRP has jurisdiction over fewer research studies than in the past. Based on an informal review of a sample of institutions, approximately 74% of domestic institutions currently holding an FWA that formerly held a Multiple Project Assurance (MPA) apply either subpart A or subparts A, B, C, and D of 45 CFR 46 to all research regardless of support. On the other hand, greater than 90% of those same institutions applied 45 CFR 46 to all research regardless of support when they held MPAs.

Along with that finding, the authors "describe [their] review of 235 compliance oversight determination letters that the Office for Human Research Protections (OHRP) issued to 146 institutions between August 1, 2002, and August 31, 2007." Such a large dataset would seem to be a rich source of information for those interested in how researchers and institutions go wrong.

Unfortunately, determination letters are generally opaque documents that fail explain the events leading up to a complaint. For example, OHRP's March 2007 letter to New Mexico State University tells us a bit about office procedure at New Mexico State, but nothing about the design of a project called "The Impact of Education in Navajo Nation Border Community Public Schools on the Hearts, Minds, and Spirits of Navajo Students," or whether that project violated any research ethics.

Because of this opacity, a review of determination letters can tell us that 56 percent of institutions were cited for problems with the "IRB initial review process" while only 3 percent were cited for misapplication of exempt categories of research. But it can't tell us what percentage of citations concerned ethnographic fieldwork, what percentage involved graduate students, or what percentage involved questions of whether data were publicly available. Nor can it tell us whether any one of these 235 letters documented actual harm to a research participant.

(I am puzzled by the finding that OHRP issued 260 citations for deficient "IRB-approved informed consent documents/process" but only 19 for "Failure to obtain informed consent of subjects." Does that mean that deficient processes still work 93 percent of the time?)

OHRP is missing an important opportunity. When courts rule on cases, they take care to explain not only the legal principles involved, but also the underlying facts, so that other courts can understand which principles should be applied to certain fact patters. By failing to document the substantive concerns that led to its investigations, OHRP has failed to educate researchers and IRBs about what kinds of research are most likely to lead to trouble.

OHRP Continues Indiana-Bloomington Investigation

In September I reported on the OHRP investigation of Indiana University-Bloomington, ably covered by the Bloomington Herald-Times. Along with several stories, an editorial, and at least two op-eds, the newspaper posted heavily redacted copies of the OHRP letter and the IU reply.

On October 3, I submitted a Freedom of Information Act request for the unredacted OHRP complaint. This week, I received a reply, dated November 17, stating that "the subject matter of your request is the subject of an open and ongoing investigation. Release of any additional information at this time could reasonably be expected to interfere with ongoing proceedings." Hence, I received no information. We still don't know what Indiana-Bloomington did to bring down the federal hammer, or if the hammer will strike again.

Meanwhile, the crackdown has disrupted research, especially for social scientists. As the Herald-Times reported on October 8 (Nicole Brooks, "IU Research Oversight Office Has More Staff, But Projects Still Delayed"),

The need to “attend to the compliance issue speedily” led to Bloomington researchers using the same proposal forms as IUPUI faculty, according to [Research Affairs Committee chairman Stephen] Burns. These forms are designed for medical research, and are “more complex than needed,” especially for social science researchers.

This has caused some faculty — and students — to not bother with some research projects, Burns said. And some students are changing their thesis topics so they don’t include human subjects research, he said.

Before the compliance issue came into play, when Bloomington campus faculty used their own form and not IUPUI’s, this was a problem, Burns said. Research topics are becoming more and more diverse, and the divide between what information is necessary for different kinds of research is widening, he said.

The upshot is that faculty and students at a major research university are abandoning their research because of secret allegations against their university's administration. It's all very well for OHRP to claim (as Ivor Pritchard did at the October SACHRP meeting) that OHRP enforcement actions are rare. But even the rare crackdown, if as severe as this one, is enough to have IRBs nationwide quaking in fear, and putting regulatory compliance above all other considerations.

Crackdown at Indiana University

The Bloomington Herald-Times reports (August 10-12) on problems with human subject reviews at Indiana University in Bloomington (IUB). Though the paper does not give details of what went wrong, it does state that in the summer of 2008, two whistleblowers in the human subjects office successfully the appealed negative evaluations they had received after airing complaints. Their immediate supervisor, Carey Conover, has been reassigned, But Conover's boss, Eric Swank, has been promoted to executive director for research compliance for Bloomington and other Indiana campuses, with a salary bump from $79,000 to $119,600.

Moreover, the university has moved to protect itself by layering on more administration. According to an August 17 Herald-Times column by the university provost, Bloomington's new president "has expanded the budget for research compliance by $4.3 million - the single largest addition to his budget - in order to create a university-wide organization of well over 100 people, professionals whose sole mission is to preserve and protect the university's research mission." And starting July 1, all IUB studies have been sent to the Indiana University-Purdue University Indianapolis (IUPUI) IRB, where, the university promises, they will be met with an "AAHRPP-accredted HRPP" and legions of "CIP-certified staff members." Meanwhile, Bloomington IRB members and staff were sent for reeducation by Jeffrey Cohen, who, no doubt, told them to review oral history.

None of this is reassuring to social scientists back at Bloomington. Writing in the Herald-Times on September 14, Noretta Koertge, a specialist in research ethics, urged "the university to take this opportunity to resist bureaucratic mission creep." Lower on the chain, informatics PhD student Kevin Makice frets that the dust-up will delay his research to the point that he will have to rely on theory and public data to meet a conference deadline. He writes, "The human-computer interaction crowd often goes to [the Computer/Human Interaction conference] talking about the woes of the research approval process only to hear how much simpler it is on other U.S. campuses and seemingly non-existent off the continent. Now, with IUPUI overburdened by serving multiple campuses--which apparently is in the long-term restructuring plans anyway--we miss the days of it just being too complicated."

Back in 2005, the Illinois White Paper on IRBs complained that the "death penalty" of shutting down all research at a university in response to a single IRB violation. This penalty, the paper warned, was largely responsible for IRBs' terrified emphasis on regulatory compliance. It looks like Indiana researchers will suffer for the sins of the research administrators.