Tuesday, September 30, 2008

Crackdown at Indiana University

The Bloomington Herald-Times reports (August 10-12) on problems with human subject reviews at Indiana University in Bloomington (IUB). Though the paper does not give details of what went wrong, it does state that in the summer of 2008, two whistleblowers in the human subjects office successfully the appealed negative evaluations they had received after airing complaints. Their immediate supervisor, Carey Conover, has been reassigned, But Conover's boss, Eric Swank, has been promoted to executive director for research compliance for Bloomington and other Indiana campuses, with a salary bump from $79,000 to $119,600.

Moreover, the university has moved to protect itself by layering on more administration. According to an August 17 Herald-Times column by the university provost, Bloomington's new president "has expanded the budget for research compliance by $4.3 million - the single largest addition to his budget - in order to create a university-wide organization of well over 100 people, professionals whose sole mission is to preserve and protect the university's research mission." And starting July 1, all IUB studies have been sent to the Indiana University-Purdue University Indianapolis (IUPUI) IRB, where, the university promises, they will be met with an "AAHRPP-accredted HRPP" and legions of "CIP-certified staff members." Meanwhile, Bloomington IRB members and staff were sent for reeducation by Jeffrey Cohen, who, no doubt, told them to review oral history.

None of this is reassuring to social scientists back at Bloomington. Writing in the Herald-Times on September 14, Noretta Koertge, a specialist in research ethics, urged "the university to take this opportunity to resist bureaucratic mission creep." Lower on the chain, informatics PhD student Kevin Makice frets that the dust-up will delay his research to the point that he will have to rely on theory and public data to meet a conference deadline. He writes, "The human-computer interaction crowd often goes to [the Computer/Human Interaction conference] talking about the woes of the research approval process only to hear how much simpler it is on other U.S. campuses and seemingly non-existent off the continent. Now, with IUPUI overburdened by serving multiple campuses--which apparently is in the long-term restructuring plans anyway--we miss the days of it just being too complicated."

Back in 2005, the Illinois White Paper on IRBs complained that the "death penalty" of shutting down all research at a university in response to a single IRB violation. This penalty, the paper warned, was largely responsible for IRBs' terrified emphasis on regulatory compliance. It looks like Indiana researchers will suffer for the sins of the research administrators.

Monday, September 29, 2008

AHA Comments on IRB Training

The American Historical Association has posted a copy of the comments on IRB training and education it sent to OHRP in response to the July notice in the Federal Register. The AHA letter states that historians "are concerned that the proposed training program will reinforce the tendency to treat all research as if it was conducted in the experimental sciences" and that "the proposed training program would only cover what should be assessed by the review boards, and does not include room for discerning among different types of research methods."

Wednesday, September 24, 2008

Final Comments on Training Requirements

A couple of readers told me my Draft Comments on Training Requirements
proposed an unrealistically burdensome training regime. I don't think that asking researchers and IRBs alike to learn about the documented ethical challenges of a given line of research is unduly burdensome, and neither did the architects of the present system. I have added the following to my comments, which I submitted today:


Lest all this sound like too much work, let me quote the Belmont Report's own recommendations:

the idea of systematic, nonarbitrary analysis of risks and benefits should be emulated insofar as possible. This ideal requires those making decisions about the justifiability of research to be thorough in the accumulation and assessment of information about all aspects of the research, and to consider alternatives systematically. This procedure renders the assessment of research more rigorous and precise, while making communication between review board members and investigators less subject to misinterpretation, misinformation and conflicting judgments . . . The method of ascertaining risks should be explicit, especially where there is no alternative to the use of such vague categories as small or slight risk. It should also be determined whether an investigator's estimates of the probability of harm or benefits are reasonable, as judged by known facts or other available studies.
Standardized training does not produce "the accumulation and assessment of information about all aspects of the research," it does not make explicit the "method of ascertaining risks," and, most significantly, it does not provide the IRB or the investigator with "known facts or other available studies." What it does produce is a false sense of expertise among IRB board members, leading to the very "misinterpretation, misinformation and conflicting judgments" against which the National Commission warned.

Wednesday, September 10, 2008

Draft Comments on Training Requirements

Back in July I reported that OHRP was seeking comments on its requirements for human subjects training for investigators and IRB members. The deadline for comments is September 29.

Here is a draft of of my comments. I would appreciate comments on the comments prior to the deadline.


Dear Dr. Carome,

Thank you for the Request for Information and Comments on the Implementation of Human Subjects Protection Training and Education Programs, published in the Federal Register on July 1. I would like to offer some brief comments on this issue.

In tracing the debate over IRB review of the humanities and social sciences as it developed over the past forty years, I have yet to come across anyone who suggests that scholars should conduct research without first receiving training of some sort. The whole purpose of a university is to teach researchers to form their inquiries along lines that will produce the best results, in ethics as well as knowledge. When it comes to human subjects protections, the question is what form of training will produce those results. So far, two general models have been proposed, and I would like to offer a third.

One model demands that every university researcher, regardless of her scholarly discipline or her subject of study, complete a basic, online course in medical ethics and regulatory compliance. The CITI Program, founded by a medical researcher and a medical research administrator, exemplifies this approach. The CITI Program has the great virtue of administrative convenience. A university research office can, in a single memo, declare that all investigators must complete the program, and it can easily monitor that they have done so. But it is not clear that such mandates serve the cause of ethics, particularly when researchers are not conducting medical or psychological experiments. While the program makes efforts at including non-biomedical perspectives, the sections on such disciplines as oral history and anthropology are written by people with no expertise in those fields. The result is that much of the material in those sections is irrelevant, inaccurate, or highly dubious in its interpretations. Such programs also reduce complex ethical problems to simplistic statements to be chosen on a multiple-choice test. While I cannot offer published citations or hard data, I know anecdotally that the requirement to complete such training breeds contempt for the whole review process in many researchers.

In 2002, the Social and Behavioral Sciences Working Group on Human Research Protections pioneered an alternative approach. Rather than preparing the same curriculum for all fields, it devised reading lists specific to each discipline. For example, materials prepared for the American Sociological Association included that association's code of ethics and essays written by sociologists (see http://www.aera.net/humansubjects/courses/asa_notebook.htm). Scholars asked to complete such training are likely to take it far more seriously than a program whose medical origins cannot be disguised. On the other hand, devising a single training regime for an entire discipline will still subject some researchers to a great deal of irrelevant material. For example, an ethnographer may not need nuanced instructions on forming survey questions, nor a survey researcher instructions about participant observation.

Finally, I would like to propose a third model that goes beyond the Working Group's approach. When scholars describe the training that most influenced their ethical decisions, they are less likely to cite general codes and principles than the work of other researchers who faced very similar challenges. Criminologist Michael Rowe put this very well in his essay, "Tripping Over Molehills: Ethics and the Ethnography of Police Work," International Journal of Social Research Methodology 10 (February 2007): 37-48. Rowe wrote,

It is the nature of ethnographic research that the principles contained in methodological textbooks or professional codes of conduct will be stretched and perhaps distorted as they are applied in dynamic situations. Since policing is unpredictable, the ethical dilemmas police researchers might face cannot be easily anticipated . . . If an absolute code of ethics is not feasible, researchers must be prepared to be reflexive in terms of ethical dilemmas and the methodological difficulties experienced in securing informed consent and meaningful access to research subjects. (48)

The best preparation for observing police work, he explains, is reading other accounts of observing police work. I believe this emphasis on specificity would hold true for most qualitative research (and a great deal of quantitative work as well).

I would suggest, then, that rather than impose universal, top-down training like the CITI Program, or even more specific top-down training like the Working Group curricula, OHRP empower researchers to devise their own ethical reading lists of materials most relevant to their work, just as they choose their own methodological models. A researcher seeking IRB certification could present an annotated bibliography, showing that he had investigated the problems he was most likely to encounter and the ways that other scholars had dealt with those problems. Researchers should also be able to use courses and seminars they have completed as evidence of their prepraration.

I assume the goal of any training requirement would be to get researchers to think seriously about the ethical problems they will face. Asking them to research those problems themselves will be far more effective than any multiple-choice test.