Thursday, December 28, 2006

The Miseducation of Jystar

Jystar, a PhD student in information and computer sciences at the University of California, Irvine, writes:

I was at the gym this morning, and on the large and annoyingly loud HD TV that I usually try to ignore was a news story, I believe it was on ABC, and badly behaved kids in a restaurant. I know, urgent breaking news, right? apparently, the station had hired two child actors to be obnoxiously loud and annoying children in a restaurant, as well as hiring an actor to play the father who ignored them and talked on his cell phone the whole time. they showed clips from hidden cameras in the restaurant in which the kids were banging on the plates and silverware, singing loudly, chasing around, diving under tables, etc. I think the purpose was to see how far they could go before someone would do something. a number of people tried to talk to the father, who just acted confused. "what do you mean my kids are out of control?" others tried to stop the kids, who either ignored them or got more annoying. when the restaurant manager came out, the kids hid under a table. why this is news is beyond me, but there's something even more troubling here.

through what process did this station have to go to get this segment approved? clearly, there are specific legal processes through which one must go to show a recording of some random person on TV. did they do a "gotcha" thing where they came out and had everyone sign forms? even beyond the legal questions, these people's dinners were almost completely ruined. and for what? for our entertainment on the morning news? if I wanted to do a similar social-behavior experiment, I would have to get it approved by my university's Institutional Review Board (IRB).
Under the First Amendment, there are, in fact, no "specific legal processes through which one must go to show a recording of some random person on TV." It is true that under some circumstances television crews and other journalists can be found liable for various privacy torts, but for the most part they enjoy great leeway to report the news, and they are not subject to prior restraint.

The IRB regime rejects the free-speech approach of relying on tort law to deter bad behavior, and replaces it with the idea that all is forbidden unless it is specifically approved. In doing so, it has robbed Jystar, and countless others, of their understanding of what it means to live in a free society.

Tuesday, December 26, 2006

Carpenter and Hyman

The Northwestern Law Review is preparing a symposium issue about IRBs. At least two papers from the symposium have appeared on SSRN.

Dale Carpenter, in "Institutional Review Boards, Regulatory Incentives, and Some Modest Proposals for Reform," writes that "IRBs understand well the physical risks to subjects that come from biomedical research. They are much less adept at identifying substantial nonphysical risks or the research methods in social science that can create them. Thus, in an abundance of caution, they 'make decisions on the basis of worst-case scenarios.' Fear of these worst-case scenarios leads them to over-regulate."

David Hyman, one of the authors of the recent AAUP report on IRBs, is more aggressive in his piece, "Institutional Review Boards: Is This the Least Worst We Can Do?" He notes that even in biomedical research, "there is no empirical evidence that IRBs have any benefit whatsoever," and that "it is hard to make the case that IRBs, with their obsession with paperwork and the tweaking of consent forms, actually promote the protection of human subjects." Both agree that the current system punishes IRB members for approving dangerous research but not for blocking legitimate research, thus encouraging boards to be over-cautious and censorious.

Both Carpenter and Hyman offer what they term "modest reforms." But many of their suggestions are anything but modest. In particular, excluding from review all interview-based research with legally competent adults (as advocated by the AAUP) seems to require either a rewriting of federal regulations (and corresponding state laws) or a willingness of institutions to risk the suspension of all research by the federal Office of Human Research Protections. I hope that legal scholars will do more to determine exactly how to change the current regulations or establish a saner and more consistent interpretation of them than the arbitrary pronouncements of the past few years.

Saturday, December 23, 2006

A thousand cuts

Many complaints about the IRB regime, such as the AAUP's recent report, focus on extreme cases where IRBs delayed or prohibited research. 10-Year-Plan, a graduate student's blog, describes a more typical problem, the loss of time even when an application is eventually approved or ruled exempt:
I spent a good deal of time this semester working on an IRB application. My dissertation research includes a good deal of oral history. Some of these are taped and archived in libraries across the country. However, most of what I am interested in, (girls' lives, gender-related issues) cannot be found in traditional archives. So, in the best-case scenario, I will be conducting oral histories myself . . .

The official word is that oral history is NOT regulated by the IRB--as long as events covered in interviews are confined to the past. However, there is some argument as to whether or non-regulated status is going to stay. In addition, the American Historical Association encourages all graduate students to clear their oral history projects just in case. In case of what, I am not certain. The chair of our department agrees and so, I embarked on the application process.

The application is long and tedious. NO, I will not inject my subjects with any substances. I am doing no testing of DNA...

Anyway, I found out today that they gave me the status of "not regulated" which means I don't have to change anything! And I don't have to re-apply next year! Happy Holidays, me.
Happy Holidays, LaKisha. The AHA was right to warn you; some IRBs would strip you of your degree if you did not follow their procedures. But your IRB has robbed you of valuable time. Had it published its criteria for what is and is not regulated, you could have avoided the time-consuming application.

Monday, December 18, 2006

Carome, Cohen, and Normal Science

Dr. Jeffrey Cohen reports the following:

At the PRIM&R conference in Washington last week I moderated a panel with Michael Carome and Julie Kaneshiro from OHRP on "When is it Human Subjects Research?" In his presentation, Dr. Carome finally clarified OHRP's position on oral history. As many of you know, in 2003 Dr. Carome wrote a letter stating that OHRP concurred with the position that oral history activities in general do not involve research as defined by the HHS regulations. Many oral historians took that to say that oral history was excluded from IRB review, including the Oral History Association. In his presentation at PRIM&R Dr. Carome clarified that this was meant in the same sense that drawing blood "in general" was not research.


Two points worth noting:

1. Dr. Carome chose to announce his latest views on historical research not to a group of historians, but to a group called Public Responsibility in Medicine and Research, a body dominated by professionals involved in biomedical research.

2. To understand the ethics of historical research, he sought an analogy in medical research: drawing blood.

In both these actions, he shows his deep grounding in biomedical research and his discomfort and unfamiliarity with other fields.

Dr. Cohen himself continues:

It is not the methodology that determines whether an activity is human subjects research, but whether it meets the regulatory definition of research - a systematic investigation designed to develop or contribute to generalizable knowledge. . . . The problem is that the regulations don't define "systematic investigation" or "generalizable knowledge" and they don't say who is to make that determination.

It is unfortunately true that the regulations, 45 CFR 46, fail to define "generalizable knowledge." But Cohen could have considered the two places where the regulations use the term, other than to define what is research. Section 46.406 twice refers to "generalizable knowledge about the subjects' disorder or condition." In a medical setting this makes sense; it distinguishes between knowledge about an individual's prognosis and knowledge about the disease itself. But the distinction is meaningless when disease is not the issue.

Thomas Kuhn writes, "in science . . . novelty emerges only with difficulty, manifested by resistance, against a background provided by expectation." (The Structure of Scientific Revolutions, 2d edition, p. 64). Both Carome and Cohen expect to see research that fits into their expectations for medical research. Faced with cases that don't fit, they try to shoehorn them into their preconceived models, ignoring contrary evidence. We need a revolution.

Tuesday, December 12, 2006

Introduction

In 1974, Congress passed the National Research Act. This act required researchers in biomedical or behavioral fields who received federal funds and who conducted experiments on humans to submit their work to the oversight of “institutional review boards,” or IRBs. These boards, established by universities, hospitals, and other institutions, were required to judge the ethics of a project before it could proceed. This step, Congress hoped, would prevent researchers from denying treatment to or otherwise harming the people who participated in their studies.

At some point (and I hope to learn more about this history), university IRBs around the nation began insisting that researchers in the humanities and social sciences also submit their projects for review. Ever since, many of those scholars have questioned the legality and wisdom of these demands. This blog is designed to inform the debate over IRBs by collecting breaking news, commentary, and background information on the subject.

I first became interested in this issue in July 2000. I had received a grant from the National Science Foundation to support my dissertation on the history of the Washington Metro. The grant required that if I involved human subjects in my research, I had to get approval from my university IRB. Since the definition of human subjects research seemed to include the oral-history interviews I was conducting, I accordingly gained approval for my research from Columbia University’s Human Subjects Research Committee. I kept that approval active until I had completed my research, which has now been published as The Great Society Subway: A History of the Washington Metro (Johns Hopkins University Press). More recently, I gained the approval of George Mason University’s Human Subjects Research Board for a series of interviews on the history of riot control.

At neither university did the review process do much to aid my research, and, on the whole, I consider the applications to have been a poor use of my time and that of the staff doing the review. Based on my own experience and that of other scholars, I am skeptical of the application of IRB oversight to non-experimental research. On the other hand, I do believe that historians and perhaps other scholars in the humanities and social sciences could learn from IRBs’ practices of mandatory training, careful documentation, and the review of consent forms. I therefore hope to find ways in which scholars in the humanities and social sciences can learn from IRB practices without being subjected to standards and practices never meant for them.