Wednesday, November 26, 2008

IRBs vs Law and Society

The Law and Society Association (LSA) has posted “The Impact of Institutional Review Boards (IRBs) on Law & Society Researchers," a 2007 report by the association's Membership and Professional Issues Committee.

In the spring of 2007, the committee put out a general call for comments from association members, receiving 24 replies. Committee members also interviewed association members, taking special interest in those who had served on IRBs. This produced some accounts of frustrating encounters that led to the destruction of research, especially cases involving research on crime and punishment, a special concern of the association.

One particularly sad story came from "Respondent 019":

I knew when completing my [questionnaire] that I would have some difficulty getting my topic approved because it related to a protected population. I completed the [questionnaire] honestly and accurately and finally heard back that I had to make substantial revisions to my proposal. Not only did I take all of the IRB’s recommendations for review into account when completing a second questionnaire, but I even had frequent contact with the compliance coordinator and the secretary to make sure I was doing everything correctly. I submitted a second [questionnaire] that covered all the areas that caused problems with my first proposal. Finally, I heard back, and I was rejected on a whole other set of criteria that IRB never mentioned when they rejected me the first time. Finally I ended up changing my topic enough so that I no longer had to deal with IRB because they had pushed back the start date of my research so far that I couldn’t risk not getting approved again.

Other accounts describe IRB blocakge of research concerning mental health facilities that house sex offenders and prisons in the United States and Turkey.

The report concludes with three sets of recommendations:

First, it wants to restrict IRB jurisdiction: "the LSA should strive to minimize the scope of the IRBs regulations over non-behavioral studies and make the procedures of approving behavioral studies as smooth and expedient as possible."

Second, the report calls for a program of research and education, ranging from conference panels and publication to a statement of best practices for research. The goal is "a nuanced and contextual view of the IRB process, one that moves away from hard and fast 'rules' for most social science research, allowing for optimal protection of human subjects without inhibiting research goals."

Finally, the report claims that "the most successful IRBs (in terms of 'customer satisfaction') are those with a decentralized 'sub-board' system," citing UCLA and Macalester as examples. I find this unpersuasive, given that the report is based on part of the work of Jack Katz of UCLA, who seems pretty unsatisfied with his IRB.

While not directly inconsistent, these three recommendations are in some tension with each other, as well as with the evidence presented in the report and positions taken by the LSA. The report's collection of horror stories--unrelieved by any reports of IRB contributions to ethical research--calls into question the propriety of any IRB review of social research, and the report's analysis suggests that there is no legal requirement for such review. Put together, these sections of the report support Katz's conclusion that "the optional decision to push all ethical review of social science and humanistic research through a prior review sieve is not only massively inefficient, it is also counterproductive where risks are most serious." ["Toward a Natural History of Ethical Censorship," Law & Society Review 41 (December 2007), 807.]

By contrast, the calls for additional research, a more flexible IRB process, and decentralization all resemble the stance of Felice Levine and others, who advocate working within the current system.

In particular, the report raises the question of LSA's position on the scope of the "behavioral research"mentioned in the National Research Act and other forms of interactions with human beings. "There has been no mention, or expressed intention regarding human subject research in law and the social sciences," the report notes, "and clearly not all research in law and society is 'behavioral.'" If this is the case, then many of OHRP's policies and interpretations of the Common Rule lack a statutory basis. Yet in December 2007, five months after the report was presented to the association, the Association signed onto Levine's comment to OHRP, in which she wrote of "social and behavioral sciences (SBS)," conflating the very categories the report wishes to keep distinct.

The July 2008 newsletter of the association mentions the report, but it does not state whether the association has accepted its committee's position or taken any action in response. I hope the LSA will continue work in this field, and that it will consider further the question of whether all survey, interview, and observational research should be made subject to a law governing "behavioral research."

Saturday, November 22, 2008

OHRP Continues Indiana-Bloomington Investigation

In September I reported on the OHRP investigation of Indiana University-Bloomington, ably covered by the Bloomington Herald-Times. Along with several stories, an editorial, and at least two op-eds, the newspaper posted heavily redacted copies of the OHRP letter and the IU reply.

On October 3, I submitted a Freedom of Information Act request for the unredacted OHRP complaint. This week, I received a reply, dated November 17, stating that "the subject matter of your request is the subject of an open and ongoing investigation. Release of any additional information at this time could reasonably be expected to interfere with ongoing proceedings." Hence, I received no information. We still don't know what Indiana-Bloomington did to bring down the federal hammer, or if the hammer will strike again.

Meanwhile, the crackdown has disrupted research, especially for social scientists. As the Herald-Times reported on October 8 (Nicole Brooks, "IU Research Oversight Office Has More Staff, But Projects Still Delayed"),

The need to “attend to the compliance issue speedily” led to Bloomington researchers using the same proposal forms as IUPUI faculty, according to [Research Affairs Committee chairman Stephen] Burns. These forms are designed for medical research, and are “more complex than needed,” especially for social science researchers.

This has caused some faculty — and students — to not bother with some research projects, Burns said. And some students are changing their thesis topics so they don’t include human subjects research, he said.

Before the compliance issue came into play, when Bloomington campus faculty used their own form and not IUPUI’s, this was a problem, Burns said. Research topics are becoming more and more diverse, and the divide between what information is necessary for different kinds of research is widening, he said.

The upshot is that faculty and students at a major research university are abandoning their research because of secret allegations against their university's administration. It's all very well for OHRP to claim (as Ivor Pritchard did at the October SACHRP meeting) that OHRP enforcement actions are rare. But even the rare crackdown, if as severe as this one, is enough to have IRBs nationwide quaking in fear, and putting regulatory compliance above all other considerations.

Friday, November 21, 2008

Can IRBs Handle the Nuances of Anthropological Ethics?

In a November 14 essay in the Chronicle of Higher Education ("New Ethical Challenges for Anthropologists"), Carolyn Fluehr-Lobban reports on the work of the Commission on Engagement of Anthropology with the U.S. Security Community, of which she is a member. The commission was established by the American Anthropological Association in 2006, in response to complaints that anthropologists had acted unethically by participating in the Department of Defense's Human Terrain System and other national security programs.

Fluehr-Lobban offers a carefully nuanced discussion of the questions of secret research and research that harms its subjects. Both require subtlely. As she notes,

even in agencies well known for their secrecy, like the Central Intelligence Agency, such terms as "transparency" and "disclosure" have become more common, and secrecy less easy to define; "classified" government documents can be accessed by scholars and journalists, while truly top-secret materials deal with intelligence research rather than anthropology. Moreover, scholars may not be able to discern whether their work contains secret material when projects are compartmentalized, and when their contribution is only a segment of a project whose wider mission is unknown. In short, anthropologists who provide "subject matter expertise" may not know the direct or indirect impact of their engagement.

Likewise, harm can be hard to predict, or even define. As Fluehr-Lobban explains,

Anthropologists have been deployed to Iraq and Afghanistan as part of Human Terrain Teams embedded with combat troops. Part of the work they do unquestionably causes harm to some people -- but it may prevent harm to others. In addition, we know so little about what the teams do, or the projects they are part of, that objective evaluation is impossible at present.

What does this tell us about IRB review of social science research? Well, that's not clear either.

On the one hand, the commission asks anthropologists to "be assured that adequate, objective review of the project has been conducted, ideally by external reviewers."

On the other hand, Fluehr-Lobban suggests that such review would require expertise not found on the typical IRB. She concludes,

Ideally, decision making occurs in a group process where the relevant disciplinary, cultural, and government-agency stakeholders are at the table . . . Consultation with professionals in related disciplines who have been grappling with issues of engagement -- for example, psychologists who have debated their role in identifying what would constitute "soft torture" and their alleged involvement in interrogations in Abu Ghraib and Guantánamo -- is also recommended, as well as with those who have been historically engaged without serious controversy -- for example, political scientists working as consultants on terrorism for defense and intelligence agencies.

Perhaps the best advice will come from one's own disciplinary colleagues. To that end, a group, "Friends of the Committee on Ethics," may be set up to offer informal, private advice about research ethics. As we think through the various issues that secrecy and doing no harm demand of us, such a committee will have an important role to play in helping define increasingly complex anthropological practice.

These latter recommendations do suggest a role for interdisciplinary consultation, but they are no endorsement for the current IRB system, which makes no provision for assuring review by relevant stakeholders, professionals who have grappled with the issues, or disciplinary colleagues. I hope that Fluehr-Lobban's commission will explore the implications of its findings for the appropriate mechanisms of ethical review.

(Thanks to the Research Ethics Blog for bringing this to my attention.)

Saturday, November 15, 2008

Comments Oppose New Regulations on Training

As reported by the Chronicle of Higher Education, OHRP recently released the replies it had received in response to its July call for comments on education and training requirements. I thank Chronicle reporter David Glenn and OHRP associate director Michael Carome for supplying me with copies of the comments.

As I see it, the comments pose three main questions.

Tuesday, November 4, 2008

Chronicle of Higher Education on OHRP Training Comments

The Chronicle of Higher Education quotes your faithful blogger in "Scholars Mull Rules for Training in Research Ethics," by David Glenn, 4 November 2008.

The story concerns the eighty or so comments received in response to OHRP's July call for comments on education and training requirements. Glenn notes that by and large, the comments were skeptical about the need for new guidance, and particularly skeptical about regulations. As he reports, "the American Association for the Advancement of Science, the Association of American Universities, the Association of American Medical Colleges, and a consortium of large social-science organizations [all] said that before the federal government issues new rules, it should carefully study whether training actually improves researchers' conduct."

I will offer some of my own comments on the comments in coming posts.