Reforms for “21st century science” would have been good for the 20th too

A group of 11 researchers and IRB professionals, most of them affiliated with the University of California, San Diego, report on a brainstorming session from early 2015. They argue that readable consent forms, expert review, a less punitive system, and more exemptions would better serve researchers and participants. While they present their ideas as “a human research protections system that is responsive to 21st century science,” the measures they propose are equally valid for research as it has been practiced for decades.

[Cinnamon Bloss et al., “Reimagining Human Research Protections for 21st Century Science,” Journal of Medical Internet Research 18, no. 12 (2016): e329, doi:10.2196/jmir.6634.]

Qualitative Sociologists Find Standard Randomness

Sociologists Sarah Babb, Lara Birk, and Luka Carfagna surveyed qualitative sociologists about their IRB experiences and heard many of the usual horror stories, from insistence on inappropriate consent forms to the dribbling out of concerns over several rounds of comments. Few of their respondents are happy with the present system, though getting the right people in key positions can help.

[Sarah Babb, Lara Birk, and Luka Carfagna, “Standard Bearers: Qualitative Sociologists’ Experiences with IRB Regulation,” American Sociologist, October 6, 2016, 1–17, doi:10.1007/s12108–016–9331-z. Note: I read a version of this article in manuscript and am so credited in the article.]

Can community partners replace IRBs for field experiments?

In her contribution to the PS symposium, Melissa Michelson argues that “real-world practitioners” will often know more about relevant ethics and law than will the members of an IRB.

[Melissa R. Michelson, “The Risk of Over-Reliance on the Institutional Review Board: An Approved Project Is Not Always an Ethical Project,” PS: Political Science & Politics 49, no. 02 (April 2016): 299–303, doi:10.1017/S104909651600024X.]

IRB Chair: "Nobody Really Knows" If IRBs Do Any Good

A former chair of the University of Wisconsin-Madison Education and Social/Behavioral Science IRB guesses that IRB review “most likely” protects subjects from harm, but concedes that “nobody really knows.” He also notes that it consumes tens of thousands of hours of work, mostly by researchers, at his university each year.

[Kenneth R. Mayer, “Working through the Unworkable? The View from Inside an Institutional Review Board,” PS: Political Science & Politics 49, no. 02 (April 2016): 289–93, doi:10.1017/S1049096516000226.]

The Cost of Ethical Review, Part III: Hindering HIV Prevention

Risk-averse IRBs are hindering potentially life-saving research, write Brian Mustanski and Celia Fisher. “Critical advances in HIV prevention among AMSM [adolescent men who have sex with men],” they note, “have been impeded by the failure of IRBs to apply federal regulations permitting adolescents to self-consent to research without parental involvement.”

[Mustanski, B., & Fisher, C. B. (2016). HIV Rates Are Increasing in Gay/Bisexual Teens: IRB Barriers to Research Must Be Resolved to Bend the Curve. American Journal of Preventive Medicine, In Press. doi:10.1016/j.amepre.2016.02.026.]

Microsoft Seeks Ethics Review Without Bottlenecks and Frustration

Duncan Watts of Microsoft Research announces Microsoft will soon launch "an ethics-review process for human-subject research designed explicitly for web-based research." Could such a process avoid the pitfalls of the IRB?

[Watts, Duncan J. “Lessons Learned From the Facebook Study.” Chronicle of Higher Education Blogs: The Conversation, July 9, 2014. h/t Rebecca Tushnet]

NRC Report: Assess Risk Empirically

One theme running throughout the NRC report is the need to replace the worthless gut reactions decried by Ezekiel Emanuel with a system that would base its judgments on the latest empirical evidence. But the report does not present a clear set of reforms that would effect this change without scrapping the current system of local IRB review.

REB Member: Make Review Process Educative, Not Regulatory

Dominique Rivière, an education researcher at the University of Toronto, reports that REB service left her somewhat less critical of the ethics-review process, but she still calls for significant reform.

[Dominique Rivière, "Looking from the Outside/In: Re-thinking Research Ethics Review." Journal of Academic Ethics 9, no. 3 (2011): 193–204. DOI: 10.1007/s10805-011-9139-y]

Can Macalester's Divisional Review Work Elsewhere?

In his contribution to the Professional Geographer special issue, Dan Trudeau of Macalester College writes that "IRBs can be a pedagogical asset, particularly if institutional review practices cultivate the habits of mind and strategies necessary for engaged and reflexive research." The key, his article suggests, may be the devolution of review to specialized committees rather than the general-purpose IRBs that are the norm. But Trudeau does not stress the degree to which Macalester's sucess depends on its departure from federal standards.

[Dan Trudeau, "IRBs as Asset for Ethics Education in Geography," Professional Geographer 64, no. 1 (2012): 25-33, DOI: 10.1080/00330124.2011.596786.]

Emmerich Reviews Behind Closed Doors and Ethical Imperialism

Nathan Emmerich of Queen's University, Belfast, finds that Laura Stark's book and my own "together . . . illustrate the nature of ethics as an aspect of research governance fundamentally contributing to our understanding of the phenomena in a manner that goes beyond the relatively limited or restricted consideration offered by applied ethical analysis."

[Nathan Emmerich, Review of Behind Closed Doors: IRBs and the Making of Ethical Research and Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965-2009, Sociological Research Online 17, Issue 2 (May 2012).]

Does the ANPRM's Appeals Provision Address IRB Incompetence?

A correspondent suggests that the ANPRM fails to address the problem of incompetent IRBs, such as IRBs that lack the expertise necessary to do their work or otherwise fail to meet all the requirements of the existing Common Rule. I am inclined to disagree.

ANPRM's Problem Statement: Helpful but Incomplete

One of the many remarkable sections of the July 26 advance notice of proposed rulemaking (ANPRM) is its admission that the Common Rule is flawed.

(Note: I have added a link to the ANPRM at the top of the link list in the sidebar.)

Since the 1970s, IRB apologists have claimed that federal regulations are flexible enough, and that local IRBs are to blame for any problems. In 2007, for example, Jerry Menikoff quoted with approval Jeffrey Cohen's 2006 claim that "the regulations provide sufficient flexibility for the efficient and appropriate review of minimal risk research. IRB review of such research does not have to be burdensome or unreasonable if IRBs appropriately utilize the flexibility in the regulations." Menikoff reiterated his claim of "flexibility within the system" in his 2009 speech, “The Legal Assault on the Common Rule."

After thirty years of such claims, it is wonderfully refreshing that the ANPRM takes so seriously many of the critiques leveled at the federal regulations themselves. And the ANPRM helpfully organizes those critiques into seven general categories.

On the other hand, ANPRM's problem statement (pages 44513-44514 in the Federal Register version) overlooks some major critiques. Fortunately, some of those critiques are implicitly recognized by some of the ANPRM's proposals.

Elliott Wants to Scrap IRBs

Carl Elliott, author of White Coat, Black Hat: Adventures on the Dark Side of Medicine, calls IRBs "incapable" and wants them replaced.

[Carl Elliott, Useless Pharmaceutical Studies, Real Harm, New York Times, 29 July 2011.]

Alarmist Views on Harvard Facebook Study

The Chronicle of Higher Ed reports a debate over a study of Facebook profiles, started in 2006 by Jason Kaufman of Harvard's Berkman Center for Internet & Society. The debate suggests that researchers may not be aware of how easy it can be to identify allegedly anonymous institutions and individuals, but that neither IRBs nor outside critics may understand all the implications of a study either.

[Marc Parry, "Harvard's Privacy Meltdown," Chronicle of Higher Education, 10 July 2011.]

SACHRP Now Lacks Social Scientists

Anthropologist Patricia A. Marshall, the sole social scientist on the Secretary's Advisory Committee on Human Research Protections (SACHRP), completed her term on the committee in March, along with three other members.

Sex Researcher Calls for "An Evidence-Informed Process"

Brian Mustanski, Associate Professor, Department of Medical Social Sciences, Feinberg School of Medicine, Northwestern University, calls for "moving the IRB process of risk/benefit assessment from being entirely subjective to being evidence-based."

[Brian Mustanski, "Ethical and Regulatory Issues with Conducting Sexuality Research with LGBT Adolescents: A Call to Action for a Scientifically Informed Approach," Archives of Sexual Behavior, published online 29 April 2011.]

Anxious Pessimism on UK's New Framework for Research Ethics

In March 2010, the United Kingdom's Economic and Social Research Council (ESRC) released its Framework for Research Ethics (FRE) as a successor to its 2005 Research Ethics Framework (REF).

David Erdos kindly alerted me to the November 2010 (Volume 15, Issue 4) issue of Sociological Research Online, which devotes a special section to essays about the new framework.

The six essays in the section suggest that British sociologists are wary of their research ethics committees and the expanded authority granted to them by the new framework.

Dreger Wants to Scrap IRBs

On the heels of Laura Stark's Los Angeles Times op-ed calling for the replacement of local IRBs with centralized boards of experts, historian Alice Dreger has published her own call for a national system of ethics review based on expertise and transparency.

[Alice Dreger, "Nationalizing IRBs for Biomedical Research – and for Justice," Bioethics Forum, 22 October 2010.]

Troubled by her IRB's approval of a project she considers unethical, and by Carl Elliott's White Coat, Black Hat: Adventures on the Dark Side of Medicine, Dreger concludes that the system of local review is ineffective:

We’ve reached the point where many people in medicine and medical ethics don’t even expect IRBs to act as something other than liability shields for their universities. But do patients who come to us only to be turned into subjects know that? Do they know that there is literally a price on their heads put there by research recruiters?

I’ve come to believe we need a radical solution. Maybe what we need is a nationalized system of IRBs for biomedical research, one that operates on the model of circuit courts, so that relationships cannot easily develop between the IRBs and the people seeking approval. This system could be run out of the Office for Human Research Protections and involve districts, similar to the federal courts system. Deliberations would be made transparent, so that all interested parties could understand (and question) decisions being made.

Think of the advantages: the possibility of actually focusing on the protection of human subjects first and foremost, free of conflicts of interest; the possibility of having nothing but trained professionals (not rotating unqualified faculty and staff) sitting on review panels; the possibility of marking biomedical research as clearly different from the social science and educational research unreasonably managed by many IRBs; the possibility of much greater transparency to those interested in seeing what’s going on; the possibility of having multi-center trials obtain a single approval from one centralized IRB, rather than trying to manage approvals from multiple local institutions. And the possibility of shutting down the deeply opaque, highly questionable private IRBs Elliott describes as being increasingly used by universities. (Go ahead, call me a Communist for caring about the Common Rule.)

Her Communist leanings aside, I don't know why Dreger presents her argument as a defense of the Common Rule, which fails to distinguish between biomedical and social research, puts ethics review in the hands of rotating unqualified faculty and staff, and keeps deliberations opaque. But her wish for the kind of coordination and transparency provided by the court system has a long lineage. I've quoted it before, and I'll quote it again:

The review committees work in isolation from one another, and no mechanisms have been established for disseminating whatever knowledge is gained from their individual experiences. Thus, each committee is condemned to repeat the process of finding its own answers. This is not only an overwhelming, unnecessary and unproductive assignment, but also one which most review committees are neither prepared nor willing to assume.

[Jay Katz, testimony, U.S. Senate, Quality of Health Care—Human Experimentation, 1973: Hearings before the Subcommittee on Health of the Committee on Labor and Public Welfare, Part 3 (93d Cong., 1st sess., 1973), 1050].

It is not lack of good intentions or hard work that leads IRBs to restrict ethically sound surveys while permitting unethical experimental surgery. It is the ignorance and isolation identified by Katz in 1973 and still in place today.

Hospital IRB Forbids Interviews with Patients

The Yale Interdisciplinary Center for Bioethics offers an interview project as one of its Cases in Research Ethics, which describe choices faced by hospital IRBs in Connecticut.

Case 3 concerns a nurse who was also a divinity school student, and who gained approval from her hospital's IRB to interview fifteen "hospital patients who were suffering from a progressive and/or life-threatening disease such as cancer" about their religious beliefs and practices and the role of religion in their feelings about their illnesses. Patients agreed to participate after "a thorough review of the purpose of the study, the nature of the questions and the time involved for participation."

Eleven interviews went fine. Then the twelfth patient "became agitated and demanded the researcher leave immediately. The researcher spoke with the hospital nurses and was informed that this subject had 'fallen away' from her prior religious involvement and had wondered if her malignancy was divine retribution for her lapse."

The researcher dutifully reported this as an adverse event. The IRB then reconsidered the project and voted 10 to 1 to forbid the researcher from interviewing the three additional patients.

As described in the case study, the IRB recognized that, collectively, it knew little about this kind of research. "While this IRB was routinely accustomed to addressing the standard types of adverse medical events seen in oncology drug trials, it did not consider the possibly significant adverse psychological consequences of asking these same subjects about their religious and spiritual beliefs vis-à-vis their disease."

Yet the IRB's awareness of its ignorance did not prevent it from stopping the research project. The case study does not give the reason for this decision.

Texas's All-Star IRB Report

In February 2008, the University of Texas System formed an IRB Task Force to examine ways to improve IRB operations throughout the UT System. In April 2009, that task force issued its report: "IRB TASK FORCE REPORT: Trust, Integrity, and Responsibility in the Conduct of Human Subjects Research."

I recently obtained a copy of that report. While it has not previously been posted on the web, a University of Texas official assured me that a final policy report like this is public information under Texas law. So to make this important public document freely available, I have posted a copy on my website. (See link above.)

The report offers an exceptionally thorough and thoughtful consideration of how IRBs should work at great research universities. While some of its recommendations may be inapplicable to univerisities that are not part of larger systems or do not operate a medical campus, many of the task force's procedures and recommendations offer a model for others. I salute all those who were involved in the report's preparation.

In particular, I commend the following elements of the UT task force and its report.

Procedures

The UT system carefully set up the task force in a way that would build respect for its findings. In particular, it managed to:

1. Represent Multiple Disciplines

UT included task force members "representing a variety of academic disciplines," among them SACRHP member Lisa Leiden. (3, 24) It also sought help from a range of consultants, including such "national experts" Tina Gunsalus; John Heldens, Moira Keene, Dan Nelson, Ivor Pritchard, and Marjorie Speers. (8)

2. Include Stakeholders

At each UT campus, the task force solicited comments from university officials, IRB chairs, investigators, and other interested parties. (8)

3. Allow Adequate Time

The UT task force completed its deliberations over the course of a year, from February 2008 to February 2009, then released its report in April 2009. A report with so broad a scope cannot be rushed.

Recommendations

Careful investigation led to thoughtful recommendations. The report suggests that the UT system:

1. Employ Faculty Expertise

The UT report recognizes that researchers are often expert in a particular area of human subjects research, and it recommends that faculty experts be identified to prepare standards for specific types of research (e.g., research involving subjects with impaired capacity, internet research) and to consult on individual projects as needed. (10)

2. Utilize Flexibility and Empirical Evidence

The report recommends that the university encourage IRB staff to provide "an efficient level of regulatory review compatible with adequate protection of human subjects," rather than the most stringent level of review. In particular, it suggests that IRB staff and members employ empirical evidence when determining risk, by consulting experts or scholarly literature. (11) The report also suggests that IRBs rely strongly on experts when determining the scientific soundness of a proposal. (14)

3. Uncheck the Box

The UT report notes that, according to Speers, fewer than 50 percent of AAHRP accredited institutions check all the box on their federal-wide assurances, and that it is "primarily the major research universities that are considering unchecking the box," thus maximizing their flexibility in handling projects not directly funded by a Common Rule agency. (12) (More on this in an upcoming post.)

4. Diversify the IRB

The UT report recommends that "Institutional Officials should ensure that the institution's disciplines are well represented in the IRB." (18)

5. Provide IRB Oversight

The UT report recommends that institutions "consider the implementation of a research ombudsperson to increase the opportunity for rapid resolution of issues involving human subjects research." (21) It also suggests an ongoing "IRB Advisory Group" to implement recommendations and assist with human subject policy issues. (23)

6. Define Key Terms

The UT Report includes definitions of key terms. Among other things, these definitions make clear that information-gathering interviews, service surveys, and classroom activities may not meet the definition of human subjects research, and that biography and oral history interviews do not meet that definition. (27) As I mentioned earlier, this recommendation has already led to the deregulation of oral history at the University of Texas at Austin.

7. Explore Alternatives

The report include an appendix on "alternative IRB models," based on the November 2006 "National Conference on Alternative IRB Models." While it is promising that the task force is willing to consider such alternatives, I would have liked it to pick up on that conference's call for "further exploration" of models for social and behavioral research. The University of Pennsylvania policy on research in the sociobehavioral sciences might be such a model.

Goal

Most importantly, the UT task force understood the IRB problem as something larger than an administrative challenge. While it explored ways to increase IRB "effectiveness, efficiency, and productivity," it went beyond such managerial concerns to probe big questions about "IRB authority, mission, and functions," the "unnecessary obstruction of research and lapses of effective human subject protection," and best practices from the available literature. (8)

The task force hoped to foster a "culture of conscience" rather than a "culture of compliance," and it understood that conscience cannot be dictated from above. (20) If other universities also seek to promote a culture of conscience, they must give a voice to all those involved with human subjects research.