Must Employees Consent?

Two recent items concerning IRBs and studies of job performance caught my eye.

The more prominent item was a December 30, 2007, New York Times op-ed, "A Lifesaving Checklist" by Dr. Atul Gawande. Gawande describes efforts by the Johns Hopkins University and the Michigan Health and Hospital Association to offer five-step checklists to intensive-care providers, to help them remember to wash their hands and take other steps to avoid infection. OHRP shut the project down, even though it had been ruled exempt by the Hopkins IRB. (So much for local control.)

Although I try to keep this blog focused on issues concerning the humanities and social sciences, and to avoid issues concerning strictly medical research, I was struck by OHRP's determination that "the subjects of the research were both the healthcare providers at the participating ICUs and their patients." Are healthcare providers really human subjects in a case like this?

A similar question arose in a non-medical setting a few weeks later. In her pseudonymous blog on Education Week, eduwonkette complained that New York City teachers were being studied without their consent and without the approval of an IRB. Since eduwonkette herself concedes that "teachers did not need to consent in this case, as they are government employees and their employers can collect whatever data they want," I don't understand why she thought an IRB should be involved. But it does suggest that the question of studying employee effectiveness is not limited to medical contexts.

Poking around in my research notes, I find that at the July 1977 meeting of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Dr. Karen Lebacqz raised just this issue as a hypothetical:

"Suppose you want to do a study that involves changing the delivery of services in a mental institution ward. The patients in the institution are the ones that you want to study in terms of what the impact is on their recovery rates and so on. The staff who work on that ward will also be very directly impacted by the research. Do they constitute subjects, even though the design and the purpose of the research is not set up to study the impact on them of what you are going to do?"

While Albert Jonsen initially suggested that the staff would constitute subjects, other commissioners disagreed:

Lebacqz. “There is a very direct impact on the licensed staff.

Ryan. “Then the administrator can force it on them as a condition of employment.

Turtle. “Yes, I was going to say, that is the whole key, it is a condition of employment. Many things that, while I am a great libertarian here, lots of things I ask the people who work for me to do, I don’t ask them for their informed consent.”

[National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, Transcript, Meeting #32, July 1977 (Bethesda: The Commission, 1977), 133-135.]

After this exchange, the commission moved onto another subject, suggesting a consensus that employees are not human subjects whose informed consent is required. In its recent determination letters on the checklist project, OHRP has ruled the opposite.

I don't expect the compliance officers at OHRP to be familiar with the commission's deliberations, and I fault the commission for not defining human subjects more carefully. But it's worth noting how far OHRP has strayed from the commission whose work nominally informs its decisions.

How IRBs Decide--Badly: A Comment on Laura Stark's "Morality in Science"

Laura Stark's recent essay in Law & Society Review led me to her 2006 Princeton University dissertation, "Morality in Science: How Research Is Evaluated in the Age of Human Subjects Regulation." The heart of the dissertation is her description of the workings of three university IRBs--one in a medical school and two at universities without medical schools--based on recordings of their meetings and her direct observation of the IRBs at work. It makes for fascinating reading, and I applaud Stark for her achievement even as I disagree with her conclusions.

Stark claims to be neutral about IRBs' ability to perform their stated task: protecting the subjects of research. She writes, "My goal is not to judge the 'fairness' and 'effectiveness' of IRBs myself." (7) And she correctly notes that the ethical acceptability of an IRB-approved project is a "social truth," not an empirical one. (244) But her tone is generally sympathetic to the IRBs. For example, she writes that IRBs' "forms of evaluation provide directed, tangible ways for board members to carrying out their review process, given the practical difficulty of applying unmediated, abstract ethics principles," making the IRB members sound like heroes who have achieved a workable system against the odds. (186)

Indeed, in some cases she reports, IRBs seem to be doing some good. For example, a physiologist and a nurse had a fruitful debate about the need for a quick medical screening of subjects in an exercise study (197-200). That's an example of an IRB with multiple experts on a single type of research--something I hope is reasonably common in much biomedical research. But most of Stark's observations are distressing in ways I don't think she appreciates. Here, then, are some of the actions she observed, along with my reasons for finding in them an indictment of the IRB system as presently run.

Johns Hopkins Dean: Generalizable Means "Hypothesis-Based"

In his comments on Inside Higher Ed's recent story on oral history and IRBs, Eaton Lattman, Dean of Research at Johns Hopkins Krieger School of Arts and Sciences, writes that "many oral history projects probably do not meet the OHRP definition of research, and are therefore free from the need to go to an IRB for approval. They are not hypothesis-based projects and they do not produce generalizable results."

This interpretation of generalizability is consistent, as best I can tell, with the intent of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research, which first established generalizability as part of the definition of human subjects research. It also resembles Columbia University's recent statement on oral history.

Dean Lattman's comment contrasts with definitions put forward by Michael Carome of OHRP and consultant Jeffrey Cohen, who maintain an oral history project is generalizable research if it lacks a hypothesis but draws conclusions, informs policy, or creates an archive for use by future researchers.

Thus, Hopkins joins Columbia and Northwestern in explicitly freeing oral history--as practiced by oral historians--from IRB review. While such advances are welcome, more welcome still would be a statement along the same lines from OHRP itself.

How State IRB Laws Threaten the Social Sciences: A Comment on Shamoo and Schwartz

Dr. Adil E. Shamoo of the University of Maryland School of Medicine kindly sent me a copy of his new article, "Universal and Uniform Protections of Human Subjects in Research," The American Journal of Bioethics 7 (December 2007): 7-9, co-authored with Jack Schwartz of the Maryland Attorney General’s Office.

The article calls for a federal law "to require that all human subject research in the United States, regardless of funding source or relationship to FDA marketing approval, be undertaken only after IRB review and with the informed consent of subjects." Barring that, it applauds state laws to that effect, such as the one passed in Maryland in 2002, which states that "a person may not conduct research using a human subject unless the person conducts the research in accordance with the federal regulations on the protection of human subjects . . . notwithstanding any provision in the federal regulations on the protection of human subjects that limits the applicability of the federal regulations to certain research . . . ." (13 Maryland Code §13-2002)

Having read the current New Yorker's story on "Guinea-Pigging," which states that 70 percent of drug trials now take place in the private sector, I can see the reason to regulate biomedical research through the state's police power, rather than through funding restrictions. But aside from a quick mention that research sponsored by the National Endowment for the Humanities is not subject to the Common Rule, the article does not mention the law's effect on the humanities and social sciences. This is a bit surprising, given Shamoo's earlier argument in the Chronicle of Higher Education that human subjects regulations "have handicapped researchers whose work poses no threat to humans." This being the case, why would he want to extend those same regulations to countless researchers not previously covered by them? And do the authors expect all journalists and oral historians in the state to be subject to IRB review, the way that many university-affiliated journalists and oral historians are now?

I wrote to both authors, and here is what I learned from them:

* Shamoo does not believe that the law requires Maryland journalists to seek IRB approval of their work.

* Schwartz does not believe that the law requires Maryland oral historians to seek IRB approval of their work, relying on a 22 November 2005 message from Michael Carome of OHRP.

* According to Schwartz, "The Maryland law, like every bill passed by the Maryland Legislature, was reviewed for constitutionality. This overall review did not address whether some hypothetical application of the law, under a specific set of facts, might raise First Amendment or other constitutional concerns."

* According to Schwartz, the Maryland attorney general's office has not brought any enforcement actions since the law's passage in 2002.

This last point is the key; a law that lies dormant for five years is unlikely to have any effect on anybody, so perhaps Shamoo and Schwartz have reason to think it will not handicap researchers more than they already are handicapped by existing federal regulation. Indeed, the legions of journalists, book authors, and market researchers who conduct interview and survey research outside of universities may have little to fear from a law that has not been enforced and might well fail constitutional scrutiny if it were.

But Shamoo and Schwartz ignore the potential effect of the law on social science and humanities researchers within universities, the very ones for whom Shamoo earlier expressed concern. These researchers have led a growing movement to get their universities to agree to review only research funded by the federal government. Such a move would free few, if any, university-affiliated biomedical researchers from oversight, since their expensive research is generally federally funded. Rather, it would free up the very kind of research that Shamoo considers too low-risk to merit review. But if the state imposes the same regulations regardless of the source of funding, university IRBs can close this door.

Thus, if the law has any effect at all, it will not be what Shamoo and Schwartz claim: "universal application of the ethical standards applicable to human subjects research." Instead, it will be the continued regulation of research by university-affiliated researchers while non-affiliated researchers conducting the same activities work in freedom. The result will not be uniformity, but entrenched disparity.

Shamoo's heart is in the right place. As he wrote in his Chronicle piece, "I have long advocated the creation of universal rules for all human-subject research, whether or not it receives federal funds. But an equally urgent reform is to exempt from present and future regulation any research that poses little or no risk to human subjects." Unfortunately, the state of Maryland has not treated exemption of social research as "equally urgent" to the universalization of rules. Thus, the effect of the law, if any, is to eliminate a path to the very type of exemption that Shamoo champions. Once again, regulators write rules in response to serious concerns about medical experimentation, with little or no attention to the social sciences and humanities.

Inside Higher Ed Reports on Comments to OHRP

Scott Jaschik's January 3 Inside Higher Ed story, "Threat Seen to Oral History," reports on the comments submitted to OHRP by the AHA and by me. The story is a helpful summary of key issues, and the comments following it indicate the passion this issue arouses in scholars in a range of fields.

Columbia University Grants Oral History Exclusion

Mary Marshall Clark, director of Columbia University's Oral History Research Office, has announced on H-Oralhist that the university yesterday approved a new policy on IRB review of oral history research. The policy notes that

Oral history interviews, that only document specific historical events or the experiences of individuals or communities over different time periods would not constitute "human subjects research" as they would not support or lead to the development of a hypothesis in a manner that would have predictive value. The collection of such information, like journalism, is generally considered to be a biography, documentary, or a historical record of the individual's life or experience; or of historical events. Oral history interviews of individuals is not usually intended to be scientific or to produce generalizable information and hence is not usually considered 'research' in accordance with the federal regulations or CU policy. Therefore, such oral history activities should not be submitted to the CU IRB for review.

Still covered by IRB jurisdiction are psychological studies that borrow some oral history techniques to test hypotheses. An example might be Kim T. Buehlman, John M. Gottman, and Lynn Fainsilber Katz, "How a Couple Views Their Past Predicts Their Future: Predicting Divorce from an Oral History Interview," Journal of Family Psychology 5 (March/June 1992): 295-318.

I hope Columbia will prove a model for other universities; in lumine tuo videbimus lumen.

American Historical Association Asks for Oral History Exclusion

In its response to OHRP's solicitation of comments on its 1998 guidance, the American Historical Association asked that "'oral history' . . . be removed from category 7 and explicitly removed from IRB review." The AHA blog, AHA Today, reports the request and posts the full text of the association's letter to OHRP.

Law & Society Review, continued

As I noted earlier, the December 2007 issue of Law & Society Review features five items concerning IRBs and the social sciences.

Malcolm M. Feeley, "Legality, Social Research, and the Challenge of Institutional Review Boards"

The section on IRBs begins with Malcolm M. Feeley's 2006 presidential address to the Law & Society Association. Feeley presents an impassioned critique of IRBs, complaining, "in the name of minimizing risks, IRBs subject researchers to petty tyranny. Graduate students and junior scholars are particularly likely to be caught in their web—and for them IRB tyranny is often more than petty. Senior scholars are generally more adept at avoidance, evasion, and adaptation, but they too are hardly exempt from this tyranny. A number of prominent social scientists, including some members of this Association, know all too well the harms of running afoul of campus IRBs. . . Entire research areas and methodologies are in jeopardy, insofar as the difficulties of obtaining IRB approval affect research priorities for funding agencies and universities' willingness to support researchers.”

Feeley then raises a number of specific problems, such as the ill fit between the beneficence encoded in regulation and the kind of social research that aspires to produce "tarnished reputations and forced resignations" of evil-doers.

To remedy this situation, Feely proposes three modes of action:

1. "Join [IRBs]; subvert them—or at least curtail them. Serve on them and do all you possibly can to facilitate the research of your colleagues rather than act as a censor."

2. Follow Richard Schweder's call to get your university to apply federal regulations only to federally funded research.

3. "Ask about estimates of how much actual harm to subjects in social science research has been prevented by IRB actions. And ask for documentation."

I am a bit skeptical about the first suggestion, for two reasons. First, few universities have IRBs strictly for the social sciences. This means that a sociologist, anthropologist, political scientist, or historian would spend most of her time on an IRB reviewing (or abstaining from reviewing) psychological experiments. That's an unfair price to pay to have some power over one's own research. Second, it assumes that IRBs are run by IRB members. As Caroline H. Bledsoe et al. report in "Regulating Creativity: Research and Survival in the IRB Iron Cage," the size of human protections staffs has ballooned in recent years. If the staff have the real power, IRB members will have little chance to facilitate research.

Laura Stark, "Victims in Our Own Minds? IRBs in Myth and Practice."

The first comment is Laura Stark's. It draws in part on Stark's 2006 Princeton dissertation, "Morality in Science: How Research Is Evaluated in the Age of Human Subjects Regulation." I am glad to learn of this work, and I hope to comment on it in a later post.

Stark suggests trying to improve, rather than restrict, IRBs, because “ethics review in some form is here to stay because of institutional inertia, and [because of her] belief as a potential research subject that ethics review is not an entirely bad idea, even for social scientists.” She advocates "changing local practices to suit the local research community, rather than refining federal regulations."

One intriguing example is the establishment of "IRB subcommittees, which can review lower-risk studies [and] have moved ethics review into academic departments. In so doing, these subcommittees of faculty members (who presumably understand the methods in question) have taken over the task of evaluating low-risk studies from board administrators." This sounds a lot like the departmental review that the AAUP suggested as an alternative to IRB control, and like the Macquarie model I described in August. I hope that Stark will publicize the name of the university that uses such subcommittees, so that it can better serve as an example to others. Stark does not explain why this model is appropriate only for low-risk studies. It seems to me the higher the risk, the more reason to have research reviewed by people who understand its methods.

Significantly, neither in her article nor in her dissertation does Stark take up Feeley's challenge to document cases in which IRBs have prevented actual harm to participants in social science research. Her research offers important insights about how IRBs reach decisions, but no evidence that those decisions do more good than harm, or that they are consistent with norms of academic freedom.

Finally, Stark claims, "the social science victim narrative—by which I mean the story that human subjects regulations were not meant to apply to us—is pervasive among academics, and it is particularly central to qualitative researchers as a justification for their criticisms of IRBs. Yet this victim narrative does not stand up to historical scrutiny, as I have shown." Yes and no. Stark's use of the passive voice (were not meant to apply) is telling; the question is who meant the regulations to apply to social scientists, and who did not. I am working on a full-scale history of the imposition of human subjects regulations on the social scientists, and I can tell Stark that more scrutiny will complicate her story.

Robert Dingwall, "Turn off the oxygen …"

The second comment is Robert Dingwall's "Turn off the oxygen …," the oxygen here referring to the legitimacy granted to IRBs by university faculty.

Dingwall is skeptical of legal challenges, given the cost, the possibility of failure, and the fact that the First Amendment only applies to the United States (Dingwall works in the UK.) He argues instead that “if we can show that ethical regulation does not actually contribute to a better society, but to a waste of public funds, serious information deficits for citizens, and long-term economic and, hence, political decline, then we may have identified a set of arguments that might lead to a more skeptical approach to the self-serving claims of the philosopher kings who sustain that system.” For example, we must continue to document ethical wrongs like the insistence by a British medical journal that two historians falsify the names of their oral history narrators, despite the wishes of most of the narrators to be named. [Graham Smith and Malcolm Nicolson, "Re-expressing the Division of British Medicine under the NHS: The Importance of Locality in General Practitioners' Oral Histories," Social Science & Medicine 64 (2007): 938–48.] I hope Professor Dingwall has a chance to read Scott Atran's essay, "Research Police – How a University IRB Thwarts Understanding of Terrorism," posted on this blog in May. It is an excellent example of the way that IRB interference can disrupt vitally important work.

Jack Katz, "Toward a Natural History of Ethical Censorship"

The third comment, by Jack Katz, is the most shocking, for it is the most thoroughly documented. (It even cites this blog, thanks.) Katz lists several cases, all recent, in which IRBs have derailed potentially important social research. Unlike the 2006 AAUP report, he gives names, universities, dates and citations for most of his horror stories. Among them:

* "In Utah, Brigham Young University's IRB blocked an inquiry into the attitudes of homosexual Mormons on their church. When the same anonymous questionnaire study design was transferred to another researcher, the IRB at Idaho State University found the study unproblematic."

* "A proposed study of university admissions practices [was] blocked by an IRB at a Cal State campus. The study had the potential to reveal illegal behavior, namely affirmative action, which was prohibited when Proposition 209 became California law."

* "At UCLA, a labor institute developed a white paper lamenting the health benefits that Indian casinos offered their (largely Mexican and Filipino) workers. Despite the university's support for the labor institute when anti-union legislators at the state capitol have sought to eliminate its funding, publication was banned by the IRB after a complaint by an advocate for Indian tribes that the study had not gone through IRB review."

Stark would have us believe that "the local character of board review does not mean that IRB decisions are wrong so much as that they are idiosyncratic." But Katz shows that IRBs' idiosyncracies can be hard to distinguish from viewpoint-based censorship.

In contrast to these identifiable harms, Katz finds "no historical evidence that the social science and humanistic research now pre-reviewed by IRBs ever harmed subjects significantly, much less in ways that could not be redressed through post hoc remedies." I don't think I would go quite this far, given Carole Gaar Johnson's description of the harms caused to the residents of "Plainville" by the inept anonymization of their town ("Risks in the Publication of Fieldwork," in Joan E. Sieber, ed., The Ethics of Social Research: Fieldwork, Regulation, and Publication (New York: Springer, 1982). But the rarity of such cases means we should weigh IRB review against other methods of prevention, such as departmental review of projects or better certification of researchers.

Katz reiterates his call, previously set forth in the American Ethnologist, for a "culture of legality," in which IRBs would be forced to explain their decisions and "publicly disseminate proposed rules before they take the force of law." He believes that "were IRBs to recognize formally that they cannot properly demand the impossible, were they to invite public discussion of policy alternatives, and were they to open their files to public oversight, they would fundamentally alter the trajectory of institutional development by forcing confrontation with the central value choices currently ignored in the evolution of ethical research culture."

But what do we do when we confront those value choices? We get statements like Stuart Plattner's: “no one should ever be hurt just because they were involved in a research project, if at all possible," a position clearly at odds with Katz's applause for "the American tradition of critical social research." (Plattner, “Human Subjects Protection and Cultural Anthropology,” Anthropological Quarterly, 2003) The problem with IRBs' value choices is not that they are hidden, but that they are often wrong. The Belmont Report is the most public and widely cited rule used by IRBs, and it is a terrible guide for the kind of critical research Feeley and Katz want done.

Feeley, "Response to Comments"

The most interesting part of Feeley's response comes at the very end. Noting that, with the AAUP's encouragement, some universities have ceased promising to review all human subjects research in favor of the regulatory minimum of federally funded research, he points out that we will soon know if the lack of IRB review of social science at those universities yields a flood of unethical research. "If there are few reports of negative consequences . . . they might encourage national officials to rethink the need for such an expansive regulatory system . . . On the other hand, if opt-out results in increased problems, the findings might help convince Katz, Dingwall, me, and still others of the value of IRBs." This strikes me a very fair bet, and the experiment can't begin soon enough.

My Comments to OHRP

As I noted in November, OHRP is soliciting comments on proposed changes to its 1998 guidance on expedited review. Below are the comments I submitted today. I thank Rob Townsend of the American Historical Association for his help in revision.

Law & Society Review

John Mueller has kindly alerted me to the December 2007 issue of Law & Society Review, which includes five items concerning IRBs. I will read and comment on them as time permits.

Readers interested in legal analysis of IRBs should also consult Philip Hamburger's "'Ingenious Argument' or a Serious Constitutional Problem? A Comment on Professor Epstein's Paper," a follow-up to the Northwestern Law Review special issue.