Symposium on Censorship and Institutional Review Boards

The long-awaited Northwestern University Law Review Symposium on Censorship and Institutional Review Boards has hit the Web. I have read and blogged about some of these articles in their SSRN incarnations, but I look forward to reading the rest.

Insider-Outsider-Down Under

David Hunter kindly alerted me to "An inside-outsider’s view of Human Research Ethics Review," posted on Culture Matters, a blog hosted by the Department of Anthropology at Macquarie University, Sydney, Australia. The posting is anonymous, but the author describes himself "as a sitting member of Macquarie University’s review board for human research," which I think identifies him as Greg Downey. Alas, another failed attempt at keeping an informant anonymous.

Downey's [?] essay is a rebuttal to Jack Katz, "Ethical Escape Routes for Underground Ethnographers," American Ethnologist 33 (2006): 499-506. In that essay, Katz argues that protocol-review, the basic tool of ethics committees, is inappropriate for ethnographic fieldwork because fieldwork is so unpredictable. As Katz puts it

when researchers participate in naturally occurring social life and write field notes on what they observe, they often encounter people and behavior they cannot anticipate. Indeed, one of the strongest reasons for conducting participant-observation research is the view that the current state of knowledge, as shaped by fixed-design research that prespecifies the kind of people to be studied and the ways to study them (sampling designs, formalized questions and protocols, and time- and space-delimited situations in which to observe), is artificial, a product not of the subjects’ social lives but of prejudice.

He also notes that some ethnographers draw from past experiences and observations of everyday life, neither of which can be reviewed by an ethics committee. He then suggests ways that researchers and universities might escape the regulatory boundaries that seem to require prior review of research.

Downey [?] seeks to rebut this argument by insisting that prior review can improve the ethical content of anthropological research. He writes,

The ethics review process should not be avoided, escaped, or ‘exempted’ away. Rather, ethics review boards can be educated about ethnographic research methods and encouraged to produce clear standards for our research. I worry that too many anthropologists inadvertently suggest that ‘ethics’ is a bureaucratic hoop, that the ‘politics of representation’ is a far more worthy consideration than the nuts and bolts of evaluating risk, minimizing dangers to participants (including researchers), balancing public interest against risks that can’t be eliminated, and thinking hard about our relationships to our subjects, our collaborators, the field, the public at large, our home institutions, and those who support our work.

This is unresponsive to Katz's critique. If anthropologists lack "clear standards for our research," by all means they should develop them, with or without the help of scholars in other fields. But I don't see how ethics committees can contribute to this effort by demanding from researchers that they get "preauthorization for observations and interviews," as Katz puts it. That's just a demand for information that doesn't exist.

Downey [?] also writes,

Katz’s suggestion that decisions be made public—for many reasons—seems to me an excellent one, but that can happen on the departmental level even without university boards being involved. That is, each student need not invent the application anew every time. The goal is not vacuous or self-righteous ‘boilerplate language’ for ethics applications, as one recent anthropology blogger suggested, but a legitimate attempt by the anthropology community to think about effective techniques for recurring issues such as oral informed consent, naturalistic observation in heavily trafficked settings, the use of photographs, the protection of populations under dangerous regimes, and the ethical requirements on those learning of illegal activity.

OK, so we have some movement toward compromise and consensus. I would like to suggest that if Downey [?] believes that departments are the appropriate organs to publicize ethics-committe success stories, the first department to do so should be the Department of Anthropology at Macquarie University. A listing of proposed ethnography projects and the improvements made to them by the Macquarie ethics committee could prove a model for researchers around the world.

Finally, I thank Downey [?] for drawing my attention to Australia's
National Statement on Ethical Conduct in Human Research. This document is so shocking that I will save comments on it until I have more time.

To the Historian All Men Are Dead

Blogger's note: Since I find myself in dialogue with a bioethicist working in the United Kingdom (see IRBs vs. Departmental Review and its many comments), now seems like a good time to present the views of Sir John Kaye, a British historian of the nineteenth century who used correspondence and interviews, as well as documents, in his work. I first read this passage as an impressionable college freshman, and it shaped my views of what historians do and why.

ZMS

Sir John Kaye, "Preface," 1870.

From Kaye's and Malleson's History of the Indian Mutiny of 1857-8 (1897-1898; reprint, Westport, Connecticut: Greenwood, 1971), vol. 2, xi-xiii.

Dealing with the large mass of facts, which are reproduced in the chapters now published, and in those which, though written, I have been compelled to reserve for future publication, I have consulted and collated vast piles of contemporary correspondence, and entered largely into communication, by personal intercourse or by letter, with men who have been individually connected with the events described. For every page published in this volume some ten pages have been written and compiled in aid of the narrative; and if I have failed in the one great object of my ambition, to tell the truth, without exaggeration on the one hand or reservation on the other, it has not been for want of earnest and laborious inquiry or of conscientious endeavour to lay before the public and honest exposition of the historical facts as they have been unfolded before me.

Still it is probable that the accuracy of some of the details in this volume, especially those of personal incident, may be questioned, perhaps contradicted, notwithstanding, I was about to say, all the care I have taken to investigate them, but I believe that I should rather say "by reason of that very care." Such questionings or contradictions should not be too readily accepted; for although the authority of the questioner may be good, there may be still better authority on the other side. I have often had to choose between very conflicting statements; and I have sometimes found my informants to be wrong, though apparently with the best opportunities of being right, and have been compelled to reject, as convincing proof, even the overwhelming assertion, "But, I was there." Men who are personally engaged in stirring events are often too much occupied to know what is going on beyond the little spot of ground which holds them at the time, and often from this restricted stand-point they see through a glass darkly. It is hard to disbelieve a man of honour when he tells you what he himself did; but every writer, long engaged in historical inquiry, has had before him instances in which men, after even a brief lapse of time, have confounded in their minds the thought of doing, or the intent to do, a certain thing, with the fact of having actually done it. Indeed, in the commonest affairs of daily life, we often find the intent mistaken for the act, in the retrospect.

The case of Captain Rosser's alleged offer to take a Squadron of Dragoons and a troop of Horse Artillery to Dehli on the night of the 10th of May . . . may be regarded as an instance of this confusion. I could cite other instances. One will suffice:--a military officer of high rank, of stainless honour, with a great historical reputation, invited me some years ago to meet him, for the express purpose of making to me a most important statement, with reference to one of the most interesting episodes of the Sipáhi War. The statement was a very striking one; and I was referred, in confirmation of it, to another officer, who has since become illustrious in our national history. Immediately on leaving my informant, I wrote down as nearly as possible his very words. It was not until after his death that I was able orally to consult the friend to whom he had referred me, as being personally cognisant of the alleged fact--the only witness, indeed, of the scene described. The answer was that he had heard the story before, but that nothing of the kind had ever happened. The asserted incident was one, as I ventured to tell the man who had described it to me at the time, that did not cast additional lustre on his reputation; and it would have been obvious, even if he had rejoiced in a less unblemished reputation, that it was not for self-glorification, but in obedience to an irrepressible desire to declare the truth, that he told me what afterwards appeared to be not an accomplished fact, but an intention unfulfilled. Experiences of this kind render the historical inquirer very sceptical even of information supposed to be "on the best possible authority." Truly, it is very disheartening to find that the nearer one approaches the fountain-head of truth, the further off we may find ourselves from it.

But, notwithstanding such discouraging instances of the difficulty of extracting the truth, even from the testimony of truthful men, who have been actors in the scenes to be described, I cannot but admit the general value of such testimony to the writer of contemporary history. And, indeed, there need be some advantages in writing of events still fresh in the memory of men to compensate for its manifest disadvantages. These disadvantages, however, ought always to be felt by the writer rather than by the reader. It has been often said to me, in reply to my inquiries, "Yes, it is perfectly true. But these men are still living, and the truth cannot be told." To this my answer has been: "To the historian all men are dead." If a writer of contemporary history is not prepared to treat the living and the dead alike--to speak as freely and as truthfully of the former as of the latter, with no more reservation in the one case than in the other--he has altogether mistaken his vocation, and should look for a subject in prehistoric times. There are some actors in the scenes here described of whom I do not know whether they be living or whether they be dead. Some have passed away from the sphere of worldly exploits whilst this volume has been slowly taking shape beneath my pen. But if this has in any way influenced the character of my writing, it has only been by imparting increased tenderness to my judgment of men who can no longer defend themselves or explain their conduct to the world. Even this offence, if it be one against historical truth, I am not conscious of having actually committed.

Study Finds IRBs Make Consent Forms Harder to Read

In "Human-Subjects Research: Trial and Error," (Nature, 2 August 2007), Heidi Ledford writes:

When [physician William] Burman, of the University of Colorado in Denver, joined in two studies run by the Tuberculosis Trials Consortium, he knew that the consent forms needed to cater to people with an eighth-grade reading level (comprehensible to an educated 13-year-old). The trials involved multiple institutions, and the forms were sent to 39 institutional review boards (IRBs) — committees designed to determine whether a proposed experiment is ethically sound. The final approvals came in 346 days later, but what the IRBs sent back, Burman found disturbing.

"The consent forms were longer. The language was more complex," Burman says. "And errors were inserted at a surprising frequency." In one case, a potential negative side effect of the treatment had been accidentally edited out. Burman responded to the problem as any researcher would: he studied it. He had an independent panel review the changes. The reviewers found that 85% of the changes did not affect the meaning of the consent forms, but that the average reading level had jumped from that of an eighth grader to that of a twelfth grader (around 17 years old)1. His results confirmed something he'd suspected for some time. "I started to think about what was happening and it just seemed like the system was flawed." It was time to change the system.

Though the article (and the accompanying editorial, "Board Games," does not mention non-biomedical research, it does highlight the problem of relying on local IRBs, which are essentially committees of amateurs, to handle specialized tasks like drafting consent forms and determining procedures for confidentiality. See In Search of Expertise.

IRBs vs. Departmental Review

In comments on this blog's introduction, bioethicist David Hunter of the University of Ulster asked me about my preferred alternative to IRB review, and I mentioned my hopes for departmental review (hopes shared by the AAUP). Lest our conversation get lost in the comments, I am moving it to this new posting:

DAVID HUNTER:

I'd disagree on departmental review being best for two reasons.

1. While a committee which has some knowledge and expertise in the area of the project, too much expertise and it becomes too close to the subject matter. This can mean that it misses significant ethical issues because they are standard practice within a specific discipline. To give one example, psychologists often want to give part of their students grade (10%) for being involved in their research. Most RECs I am involved in don't allow this practice because it is felt it is unduly coercive. I imagine if a REC/IRB was entirely composed of psychologists they may disagree.

2. It is important for a REC to be substantially independent from the researcher, but this doesn't happen in departmental review, instead the REC has an interest in the research being let to go ahead.

My university presently runs on a departmental review model, and while I can't name names I have personally seen examples of both of the above issues coming up.

I've written about these problems here:
Hunter, D. 'An alternative model for research ethics review at UK universities' Research Ethics Review. (2006) Vol 2, No 2, 47-51.
(Which unfortunately isn't available online)

and here: Hunter, D. 'Proportional Ethical Review and the Identification of Ethical Issues Journal of Medical Ethics. (2007);33:241-245.

I certainly agree with you that IRBs shouldn't be dominated by medics and medical concerns, they instead should have a wide range of representation. I'm inclined to think though that the baseline ethical issues are similar and while different rules may be appropriate for different disciplines they flow out of the same background.

In terms of examples here are a few, I can't be too specific with details for reasons of confidentiality.

1. Study of sexual attitudes in school children. Asked very probing questions as one might expect, but didn't intend to get parental consent to carry out the research a parallel can be found here: India Research Ethics Scandal: Students made guinea pigs in sex study
No consideration had been given to what might have been done if there was disclosure of harmful behaviour etc.

2. Historian was going to civil war stricken country to interview dissidents about the war, intended to publish identifying comments (without getting consent for this) which were likely to be highly critical of the current regime.

3. Social scientist wanted to understand children's attitudes towards a particular topic. As a blind so that the participant would not know the questions they wanted to answers to, they proposed to use the becks depression index. This contains questions about self harm, future worth and was potentially very distressing, not at all appropriate as a blind.

4. Student wished to conduct interviews with employees of a company on an issue that could significantly damage the companies profitability. No consideration was given to how to best report this information to minimise harm to the company.

I'm inclined to think that any sort of research involving humans can lead to harm whether that is physical, social, financial, psychological or so on. As such the benefits and the risks need to be balanced, and it needs to be considered how to minimise that harm. That I take it is the job of the researcher. However, having sat on RECs for a while it is a job that sometimes the researchers fail at spectacularly, then it becomes the job of the IRB/REC. The difficulty is how, without full review by a properly constituted REC, do you identify those applications that have serious ethical issues?

ZACHARY SCHRAG:

Thanks for these examples.

First, let me state that I am primarily interested in projects that fit Pattullo's proposal of 1979: “There should be no requirement for prior review of research utilizing legally competent subjects if that research involves neither deceit, nor intrusion upon the subject’s person, nor denial or withholding of accustomed or necessary resources.” Under this formula, the projects invovling children (who are not legally competent) and the project involving undergraduates (whose course credit is an accustomed or necessary resource) would still be subject to review.

That said, I have little confidence that IRBs are the right tool to review such research. As for child research, under U.S. regulations, and, I believe, the rules of most universities, the studies could be approved by three IRB members wholly lacking in expertise on child development. (The regulations encourage but do not require the inclusion of one or more experts when vulnerable populations are involved.) Were I the parent of a child involved in such studies (and I'm proud to say that both my children have furthered the cause of science by participating in language studies), I would greatly prefer that the protocols be reviewed not by a human subjects committee, but by a child subjects committee composed mostly or entirely of people expert in child research.

For the psychology course and the history project, the real question is whether a departmental committee can be trusted to enforce its own discipline's ethical code. The code of the British Psychological Society forbids pressuring students to participate in an experiment. And the ethical guidelines of the the Oral History Society require interviewers "to inform the interviewee of the arrangements to be made for the custody and preservation of the interview and accompanying material, both immediately and in the future, and to indicate any use to which the interview is likely to be put (for example research, education use, transcription, publication, broadcasting)." So yes, those sound like unethical projects.

Perhaps some departments would fail to correct these mistakes, just as some IRBs and RECs get them wrong. At some level this is an empirical question that cannot be answered due to the uniform imposition of IRB review. In the U.S., at least one university (the University of Illinois) had a system of departmental review in psychology that worked without complaint until it was crushed by federal regulation in 1981. With the federal government imposing the same rules nationwide, we can only guess about how well alternatives would work.

Moreover, departmental review would allow committees to bring in considerations unknown to more general ethics committees. For example, the British and American oral history codes require attention to preservation and access to recordings, something that and IRB/REC is unlikely to ask about.

I would also add that something close to departmental review is typical of the standard IRB, i.e., one in a hospital or medical school. It's true that the U.S. regulations require "at least one member whose primary concerns are in nonscientific areas" and "at least one member who is not otherwise affiliated with the institution and who is not part of the immediate family of a person who is affiliated with the institution." But the rest of the members can be biomedical researchers of one stripe or another. If that's good enough for the doctors, how about letting each social science discipline form an IRB of its members, with a community member and a non-researcher thrown in?

Still, if IRBs/RECs limited themselves to holding researchers up to the standards of the researchers' own academic discipline, I wouldn't be complaining.

Where we really disagree, then, is on project 4. You write, a "Student wished to conduct interviews with employees of a company on an issue that could significantly damage the company's profitability. No consideration was given to how to best report this information to minimise harm to the company."

That sounds a lot like this case:

Kobi Alexander's stellar business career began to unravel in early March with a call from a reporter asking why his stock options had often been granted at the bottom of sharp dips in the stock price of the telecom company he headed, Comverse Technology Inc.

According to an affidavit by a Federal Bureau of Investigation agent, unsealed in Brooklyn, N.Y., the call to a Comverse director set off a furious chain of events inside the company that culminated yesterday in criminal charges against Mr. Alexander and two other former executives. Federal authorities alleged the trio were key players in a decade-long fraudulent scheme to manipulate the company's stock options to enrich themselves and other employees.

After the March 3 phone call from a Wall Street Journal reporter, the FBI affidavit said, Mr. Alexander and the other two executives, former chief financial officer David Kreinberg and former senior general counsel William F. Sorin, attempted to hide the scheme. Their actions allegedly included lying to a company lawyer, misleading auditors and attempting to alter computer records to hide a secret options-related slush fund, originally nicknamed "I.M. Fanton." It wasn't until a dramatic series of confessions later in March, the affidavit said, that the executives admitted having backdated options. The trio resigned in May.

That's an excerpt from Charles Forelle and James Bandler, "Dating Game -- Stock-Options Criminal Charge: Slush Fund and Fake Employees," Wall Street Journal, 10 August 2006. As far as I can tell, Forelle and Bandler made no effort to minimize the harms to the companies they studied or the executives they interviewed. Their "Perfect Payday" series won the 2007 Pulitzer Prize for public service.

Your insistence that an interviewer minimize harm is a good example of an effort to impose medical ethics on non-medical research, and a good reason to get RECs away from social science.

A Biomedical Scientist Speaks Out

Adil E. Shamoo, a professor of biochemistry and molecular biology at the University of Maryland School of Medicine and editor in chief of the journal Accountability in Research, complains that human subjects regulations "have handicapped researchers whose work poses no threat to humans." ("Deregulating Low-Risk Research," Chronicle of Higher Education, 3 August 2007; thanks to John Mueller for the heads-up.)

Schwetz Retires

Dr. Bernard Schwetz has announced his retirement as director of the Office for Human Research Protections (OHRP). Schwetz has led the office since February 2003. During that time, it has agreed to broad exemptions from IRB review for oral history, reneged on that agreement by publishing nonsensical guidance, then refused to discuss the matter with historians. Most recently, in February Schwetz told the New York Times that he intended to issue new guidelines by the end of 2007, a promise that now seems unlikely to be kept.

OHRP Reprimand Puts Forms Over Substance

The federal Office for Human Research Protections (OHRP) has reprimanded the University of California at Berkeley's Committee for Protection of Human Subjects (CPHS) for what it considers misconduct in approving several projects. For a blog on the humanities and social sciences, the most relevant project was an economic study of Kenyan women, who were asked to record their sexual encounters, finances, and family events so researchers could understand their decision to engage in risky commercial sex.

Here's what OHRP had to say about the project:

OHRP finds that the informed consent documents reviewed and approved by the CPHS for study Sex Work as a Response to Risk in Kenya (Study 2005-5-2) failed to include and adequately address the following elements required by HHS regulations at 45 CFR 46.116(a):

(a) Section 46.116(a)(2): A description of any reasonably foreseeable risks and discomforts (i.e., risks and discomforts not described).

(b) Section 46.116(a)(3): A description of any benefits to the subject or others that may reasonably be expected from the research.

(c) Section 46.116(a)(7): An explanation of whom to contact for answers to pertinent questions about the research and research subjects’ rights (should include someone other than the investigator), and whom to contact in the event of a research-related injury to the subject. OHRP notes that CPHS approved an informed consent document that referenced a separate document containing the required contact information in lieu of requiring the informed consent document to contain such information.

Please note that HHS regulations at 45 CFR 46.116(d) require that an IRB find and document four specific criteria when approving waiver or alteration of some or all of the required elements of informed consent. OHRP finds no evidence demonstrating that CPHS found and documented such criteria prior to approving the informed consent document for this study, which did not include two elements of informed consent, i.e., 46.116(a)(2) and (a)(3), and which altered one element of informed consent, i.e., 46.116(a)(7).

Charge (c) strikes me as simple nitpicking. 45 CFR 46.116(a)(7) requires only that "in seeking informed consent the following information shall be provided to each subject: . . . An explanation of whom to contact for answers to pertinent questions about the research and research subjects' rights, and whom to contact in the event of a research-related injury to the subject." It does not state that the information must be on the "informed consent document" rather than a separate document.

Charge (b) is also doubtful. According to a paper derived from the study, Jonathan Robinson and Ethan Yeh, "Sex Work as a Response to Risk in Western Kenya," "Respondents in Round 1 (October - December, 2005) were compensated 1,000 Kenyan shillings (US$14), and respondents in Round 2 (July - October, 2006) were compensated 1,500 Kenyan shillings (US$21) for participating in the study." Does OHRP suppose that the respondents were not told in advance that they would be paid?

And then there's charge (a), complaining of a lack of a "description of any reasonably foreseeable risks and discomforts." Just what are the reasonably foreseeable risks and discomforts to a woman keeping "a daily diary (or logbook) in which she self-reported the shocks she encountered (own illness or injury, illness or injury of another household member, death of a friend or family member, menstruation, and incidence of a sexually transmitted infection), her sexual behavior with up to 3 partners each day, her income, and her expenditures [and] additional information on client characteristics and unprotected vaginal and anal sex, separately"? Or, more precisely, who is best able to describe those risks: the IRB, the investigator, or the women who are keeping the diaries and would have to live with the consequences of their somehow getting into the wrong hands? I'd say it's the women, so a list of risks provided by a couple of American economists would be useless.

As Bradford Gray wrote in 1978, there is a "distinction between informed consent and consent forms." (American Sociologist, August 1978, 163). Joan E. Sieber and Robert J. Levine expand on this distinction in "Informed Consent and Consent Forms for Research Participants," Observer, April 2004).

OHRP also complains that

CPHS conditionally approved the above-referenced study even though CPHS noted that the protocol contained little information regarding:
(i) Ensuring that risks to subjects are reasonable in relation to anticipated benefits, if any, to subjects, and the importance of the knowledge that may reasonably be expected to result (namely, sample size);
(ii) Equitable selection of subjects (namely, subject recruitment and enrollment procedure); and
(iii) Informed consent (how sought and documented).

It is not clear how the sample size would affect the ethical validity of the study. The other issues are nicely addressed in the Robinson and Yeh paper, though I can only guess what materials they submitted to CPHS in 2005. But notice that OHRP complains only that paper is missing, not that the project was ill-designed, or the participants ill-used. A true audit of the CPHS's effectiveness in this case would require going to Kenya and asking the participants in the study whether they understood the study and felt they had been treated fairly. Lacking the will or resources to do so, OHRP frets over Berkeley's failure to shoehorn a social science project into requirements devised for medical experimentation. The lesson for IRBs is that they will be judged not by the ethical content of the projects they approve, but by the adherence of consent forms to standard templates.

Berkeley's IRB is being scolded for cutting its researchers some slack, and now, I fear, other IRBs will react by becoming even stricter with social research. So much for flexibility.

White, "Institutional Review Board Mission Creep"

Ronald F. White, a professor of philosophy at the College of Mount St. Joseph, is fed up with IRB review of the social sciences: "Institutional Review Board Mission Creep: The Common Rule, Social Science, and the Nanny State," Independent Review 11 (Spring 2007): 547-564.

Much of the piece will be familiar to those who have followed recent IRB debates, but I did enjoy White's first-hand account:

My IRB experience with graduate student projects on leadership was eye opening. A colleague and I taught the course. We spent hours checking student IRB forms, and half the semester was consumed in getting their protocols past the committee chair. All of these projects involved harmless interviews and questionnaires to be done in the workplace. The overwhelming majority of the students' employers not only supported their research, but in many instances were paying for them to attend graduate school. All of my students found the IRB debacle to be nitpicking nonsense. Many of them ultimately received an "incomplete" for the course. It would be convenient simply to blame our IRB chair for this debacle. However, that person was not only a highly competent and cooperative IRB chair and an established social scientist, but also an extraordinarily cooperative friend of mine. In short, the IRB fiasco is not about persons, but about a system.

After that initial experience, the program redefined the project so that all students could get IRB approval by providing the same answers on the form. This adaptation made IRB compliance less onerous, but it severely limited the student's choice of topics and deprived them of the opportunity to do real science. Since then, the course has introduced a whole new kind of research option for students that avoids IRB involvement. I surmise that in most educational settings, the demands of IRB compliance have led to requiring topics and projects that are easier to get past boards.

There are a couple of points here. First, as Robert Kerr has noted, research delay may be research denied, so we should not take at face value IRB claims about the low percentage of projects that are rejected outright. Second, boilerplate approval processes may lead to boilerplate research--the chilling effect that IRB critics have noted for decades.

What intrigues me most is how a philosopher got snared in this mess, and I hope to learn more about the course and the research White's students were pursuing.

The History of 45 CFR 46

I am working on an article on the development of federal regulation of social science research from 1966 to 1981, and perhaps to 1991, when the current 45 CFR 46 was promulgated. I will present portions of this work at two upcoming conferences: the Oral History Association meeting in Oakland, California, October 24-28, 2007, and the Organization of American Historians meeting in New York City, March 28-31, 2008.

While I cannot offer IRB advocates a place on my panels, if anyone wishes to attend and comment, I will ask my panel chair to recognize that person.