CITI Program is not unique in its mortifying stupidity

Writing in Slate, L. V. Anderson condemns simplistic, online training programs that are supposed to encourage regulatory compliance, but really just suck up time and money without improving behavior.

[L. V. Anderson, “Ethics Trainings Are Even Dumber Than You Think,” Slate, May 19, 2016.]

Anderson writes,

Regulators, managers, and employees are caught in a vicious cycle. Regulators pressure companies to implement training programs in hopes of reducing corporate crime and malfeasance. Executives implement training programs in hopes of protecting themselves against lawsuits and prosecution. Employees see through executives’ motivations and ignore, or even rebel against, the lessons of the trainings.

Although there’s not much research one way or the other, the online nature of compliance courses probably exacerbates this vicious cycle.

Anderson does not specifically mention the mortifyingly stupid CITI Program and its cousins in the IRB world, but everything she says applies to them.

Mortifyingly Stupid CITI Training Kills Oral History Course

Writing on H-Oral, Carl Kramer, the retired director of the Institute for Local and Oral History, Indiana University Southeast, reports that Indiana University's requirement of the mortifyingly stupid CITI Program dissuaded him from requiring oral history students to conduct actual interviews:

The issue is whether the training is relevant to oral history. Last year, I planned to have the students in my Oral History course at Indiana University Southeast interview baby boomers who grew up in a nearby former company town. The local library had conducted a similar project of the previous generation about 25 years earlier, and I thought it would be a great follow up. But meanwhile, Indiana University had adopted a national training program for human subjects research that was oriented toward biomedical and psychological standards, including units on dealing with pregnant women and fetuses, HIPPA, medically-oriented conflict of interest issues. It took me approximately seven hours to review the tutorial and take the exam. As an instructor who has conducted hundreds of interviews for many years, I concluded that if it took me that long to take an exam whose content was largely irrelevant to oral history, then I could not reasonably require my students to take it. So I ended up giving them the option of doing an interview or taking a final exam. The split was about 50-50, with the majority opting for the final exam. This was for an expedited review project through the IRB. I retired from the institute under which I taught the course, and I would never again teach a course that required such an irrelevant exam.

His comment comes in reply to a posting by a Kent State graduate student who may lose grant funding because she relied on OHRP's 2003 letter stating that "oral history interviewing activities, in general, are not designed to contribute to generalizable knowledge and, therefore, do not involve research as defined by Department of Health and Human Services (HHS) regulations at 45 CFR 46.102(d) and do not need to be reviewed by an institutional review board (IRB)."

What Can One University Do?

A few weeks ago, a correspondent asked me what reforms individual universities can implement while awaiting systemic, regulatory reform. It's an excellent question, so here's a roundup from material previously covered on the blog.

No university has adopted all of these measures, and at least one of these measures has not been adopted by any. But most of them are in place already, and there's no reason they can't spread.

Report from the National Academies Workshop

Last week I attended the Revisions to the “Common Rule” in Relation to Behavioral and Social Sciences Workshop sponsored by the National Academies.

I live-tweeted the event on my @IRBblog account, and I have collected those tweets on Storify.

What follows are what I consider some of the key messages from selected presenters. The statements following each name represent my summary of the remarks, not necessarily a quotation or paraphrase.

Could Guidance and Feedback Replace Rote Compliance?

Murray Dyck and Gary Allen, both of Griffith University in Australia, argue that "the review process should be an advisory and collegial one—not one that focuses on compliance, enforcement and gatekeeping."

[Murray Dyck and Gary Allen. “Is Mandatory Research Ethics Reviewing Ethical?” Journal of Medical Ethics (August 3, 2012), DOI: 10.1136/medethics-2011-100274.]

IRB Chair Denies Being a Vampire

Patricia Price, a geographer and IRB chair at Florida International University, assures us that her board is not "a malevolent, vampirish entity."

[Patricia L. Price, "Geography, Me, and the IRB: From Roadblock to Resource," Professional Geographer 64, no. 1 (2012): 34-42, DOI:10.1080/00330124.2011.596789]

Berkeley Historian Defends IRB Review of Oral History

Martin Meeker, a historian with the Regional Oral History Office (ROHO) at the University of California, Berkeley, argues that "Historians of the recent past, many of whom use interviews as a source, need to be more systematic about doing oral histories as a form of research [and] that cooperation with IRBs offers one way to do that." What he really means, I think, is that cooperation with IRBs may help historians get legal help from their universities.

[Martin Meeker, "The Berkeley Compromise: Oral History, Human Subjects, and the Meaning of 'Research,'" in Doing Recent History: On Privacy, Copyright, Video Games, Institutional Review Boards, Activist Scholarship, and History That Talks Back, edited by Claire Bond Potter and Renee C. Romano (Athens: University of Georgia Press, 2012).]

Community Researchers Flee the CITI

An article in the latest issue of the Journal of Empirical Research on Human Research Ethics finds that standard research ethics training programs--specifically the mortifyingly stupid CITI Program--are inappropriate for Community-Engaged Research (CEnR).

[Anderson, E., S. Solomon, E. Heitman, J. Dubois, C. Fisher, R. Kost, M. Lawless, et al. "Research Ethics Education for Community-Engaged Research: A Review and Research Agenda." Journal of Empirical Research on Human Research Ethics 7, no. 2 (April 2012): 3, DOI: 10.1525/jer.2012.7.2.3]

Anthropologist: IRBs Create "the Worst of Possible Worlds"

Jonathan Stillo, a PhD candidate in medical anthropology at the CUNY Graduate Center, writes with great sensitivity of the ethical challenges he faced while studying mortally ill patients at a tuberculosis sanatorium in Romania. He suggests that IRB review was at best an irrelevance and at worst a hindrance to his efforts at ethical research.

[Jonathan Stillo, "The Trobriand Islanders Never Friended Malinowski on Facebook," 30 November 2011, CAC.OPHONY, and "Research Ethics in Impossibly Unethical Situations," CAC.OPHONY, 21 December 2011. Another hat tip to the indefatigable Roberto Veloso. I realize the IRB angle on this may be the least interesting part of Stillo's story, but this is the blog I have.]

CITI Program as Mind-Numbing, Coercive, Counterproductive McEthics

Sanjay Srivastava of The Hardest Science kindly alerted me to a newly published critique of the mortifyingly stupid CITI Program.

[Jennifer J. Freyd, "Journal Vitality, Intellectual Integrity, and the Problems of McEthics," Journal of Trauma & Dissociation, available online: 15 July 2011, DOI:10.1080/15299732.2011.602290]

Research Psychologist Blogs ANPRM

Sanjay Srivastava, associate professor in the Department of Psychology at the University of Oregon, has been posting some helpful observations about the ANPRM on his blog, The Hardest Science:

U of Michigan Reports Some Progress

The University of Michigan has released the results of a 2009 survey of investigator experiences in human research. The survey suggests that matters have improved somewhat since the university launched its HRPP Policy Innovation and Demonstration Initiative in 2007, but that more work remains to be done.

[Survey Research Center, Institute for Social Research, University of Michigan, "2009 Follow-Up Survey of Investigator Experiences in Human Research," December 2010. h/t: Human Research Protections Blog.]

The CITI Program as Mortifyingly Stupid, Marxist Doxology

The Presidential Commission for the Study of Bioethical Issues has posted videos and transcripts of its Meeting Five, held May 18 and 19 in New York City. I earlier linked to the Commission's summary of the statement by Ronald Bayer, professor and co-chair of the Center for the History of Ethics of Public Health at the Mailman School of Public Health at Columbia University. Now that we have the verbatim text, it is worth quoting as well.

Overall, Bayer lamented that the IRB system has "turned itself into an object of ridicule and sometimes contempt in a way that I think is dangerous to those who believe in the ethical conduct of research."

Particularly choice is Bayer's description of the CITI Program, a widely used online training course in research ethics, which Columbia University requires researchers to complete every three years.

University of Iowa: Ask IRB Before Researching Neanderthals

Someone at the University of Iowa apparently thinks that the IRB has jurisdiction over research with dead Neanderthals.

Survey: One-Third of UConn Researchers Dislike CITI Program

A 2007 survey of researchers at the University of Connecticut found that more than one third were dissatisfied with the Collaborative Institutional Training Initiative (CITI) program in human subjects research.

The UConn IRB and Office of Research Compliance offered the survey to about 350 researchers, of whom 114 (33 percent) returned it. Part of the survey asked respondents about the CITI Program:

7 Questions asked respondents to rate different aspects of the CITI course on a scale of 1-7 (1=least, 7=most). 4 out of these 7 questions asked if the CITI course increased understanding of risks and protections for human subjects in research. There were no statistical differences in the answers received on this group of 4 questions.

53% rated this group 5 or above
16% rated this group 4, moderate
31% rated this group 3 or below

Similar rates were received for overall satisfaction with the CITI course:

54% rated it 5 or above
9% rated it 4, moderate
37% rated it 3 or below

The course did appear to have an impact on the respondent's understanding of the Federal
Regulations. On this criteria,

72% rated it 5 or above
4% rated it 4, moderate
24% rated it 3 or below

The course had a negative impact on the respondents' willingness to join an IRB:

29% rated it 5 or above
13% rated it 4, moderate
58% rated it 3 or below

These figures suggest wider dissatisfaction with CITI than one of its founders, Paul Braunschweiger, admitted in a 2006 presentation. That presentation (slide 60) reported that principal investigators gave the program an average of about 7.8 on a 10 point scale on overall satisfaction. Though the presentation did not show the distribution of researchers' responses, it would be difficult to get so high a mean if 37 percent of researchers offered negative assessments. We need more data.

The UConn survey also offered researchers the chance to write open-ended comments. The most common suggestions were that the training should be shorter, and that the course content "should be limited to a researcher's area of research." Researchers were happy with the online form of the course, with 74 asking for no change, and only 12 choosing the next most popular option: video instruction.

All of these results suggest the potential for online courses that are shorter than CITI and targetted to a specific research discipline, such as Macquarie University's Human Research Ethics for the Social Sciences and Humanities.

UConn also surveyed researchers on their views of the UConn IRB. But the university has only reported the mean ratings, not the distribution of responses, so it is impossible to say if the IRB earned as many unsatisfactory grades as did the CITI program.