And the Bush Was Not Consumed

Robert Dingwall kindly alerted me to Scott Kim, Peter Ubel, and Raymond De Vries, "Pruning the Regulatory Tree," Nature 457 (29 January 2009), 534-535.

The authors have complex disciplinary backgrounds. Kim has an MD and a PhD in philosophy, and teaches psychiatry. De Vries has a PhD in sociology and the title of associate professor of bioethics/medical education. And Ubel has an MD and describes himself as a "physician and behavioral scientist." Together, they are unusually well prepared to think about the full range of disciplines affected by human subjects regulations.

The authors complain that IRB review of minimal risk research wastes time and money, infringes academic freedom, and threatens public health by impeding important research. They conclude that "it is unethical to support a system that creates a significant financial, scientific, clinical and ethical burden with virtually no counterbalancing good."

To fix this, they propose "a simple regulatory change that is far-reaching, equitable and yet low risk: exempt minimal-risk research from IRB review." In their scheme, researchers would still "complete a brief application describing research procedures, risks, burdens and the potential loss of otherwise expected benefits to the subjects. An institution-designated person reviews the application, and exempted protocols would not be subject to further IRB review. The application becomes the project's registration and serves as an accountability document."

This sounds fine on the surface, but the proposal raises some questions.

1. How can we change the Common Rule?

The authors propose a change in the regulations, but they do not address the difficulty of getting agreement from all Common Rule signatories. In 2002, Marjorie Speers of AAHRPP testified that " there is no effective means to [revise policy]; the agencies who are signatories to the Common Rule have not been able to make changes to it in the last 11 years even though the need for changes has existed." That was nearly seven years ago, and nothing has improved. See The Calcified Common Rule.

Under the circumstances, we may need new legislation, rather than regulation.

2. Would OHRP and institutions respond to changed regulations?

The authors recognize that even if the regulations were changed, "institutions have a tendency to impose on themselves requirements that are even more stringent than those required by law," so they might impose review on minimal-risk research anyway. Indeed, the 1998 Bell study found that less than half of protocols eligible for exemption actually received it. I imagine that percentage has gone down in the past decade, considering regulators' hostility to the existing exemptions.

To address this problem, the authors claim that "a new regulation that exempts minimal-risk research from IRB review would send a clear and unambiguous message that the government's priority is not on intense oversight of low-risk research." Perhaps, but the government does not speak with one voice, and regulators have resisted earlier calls for deregulation. The greatest effort at deregulation--the 1981 exemptions--was followed within months by OPRR suggestions that universities act more stringently than required by the regulations. The 1995 ACHRE recommendation for "alternative mechanisms for review and approval of minimal-risk studies" was followed by the OPRR crackdown that led to far more burdensome review of minimal risk research.

If the authors are serious about reform, they should seek not a "message," but rather a law that restrains OHRP and penalizes interference with minimal risk research.

3. Would exemption be any better than expedited review?

As noted above, when the authors call for "exemption," they use the term in the specialized way introduced by OPRR in 1995. Rather than meaning that a researcher is free from IRB jurisdiction, they mean that a researcher would still have to write up an application and submit it to an "institution-designated person." The authors claim that "substantial resources would be freed for better uses."

The authors base this claim in part on a 2005 finding that in US medical schools, the average expedited review cost more than the average full review ($1,060 vs. $1,021). [See Jeremy Sugarman, Kenneth Getz, Jeanne L. Speckman, Margaret M. Byrne, Jason Gerson, and Ezekiel J. Emanuel, "The Cost of Institutional Review Boards in Academic Medical Centers," New England Journal of Medicine 352 (April 28, 2005), 1825-1827.] But that study did not estimate the cost of reviews for exemptions.

Indeed, the system the authors propose is not very different from the current system for expedited review; the big difference being that the authors would delegate authority to an "institution-designated person" who might not be an IRB member. But their proposal would still require a good deal of paperwork and staff time, as well as the review by a single person, so it might not cost much less than the expedited system they seek to replace.

This question is open to empirical research. The authors, or someone else, could repeat the Sugarman analysis, but studying institutions that require staff review for proposals that are exempt under the existing 45 CFR 46.101 clauses.

4. Is pruning enough?

In sum, the regulatory tree may be tougher to prune, and to keep pruned, than the authors suppose. Here's a counter proposal adapted from advice concerning a similar growth:

For successful long term control of human subjects regulations, the extensive root system must be destroyed. Any remaining root crowns can lead to reinfestation of an area. Mechanical methods involve cutting vines just above ground level and destroying all cut material. Close mowing every month for two growing seasons or repeated cultivation may be effective. Cut human subjects regulations can be fed to livestock, burned or enclosed in plastic bags and sent to a landfill.

Blame the ESRC?

David Hunter kindly alerted me to Sarah Dyer and David Demeritt, "Un-Ethical Review? Why It Is Wrong to Apply the Medical Model of Research Governance to Human Geography," Progress in Human Geography 33 (2009).

Dyer and Demeritt attack the Economic and Social Research Council's 2005 Research Ethics Framework, the basic document guiding British Research Ethics Committees (RECs) in their oversight of social research. Some of this attack strikes me as misplaced, since it ignores elements of the Framework that address the authors' concerns.

For example, the authors complain that

in the case of critical social science, the aim of the research is typically to expose wider social injustices and in that way actually harm those who benefit from them. But, following the injunction of the ESRC (2005: para 3.2.5) that ‘[p]articipants’ interests or well-being should not be damaged as a result of their participation in the research’, it would be impossible to secure permission to interview employers whose discriminatory practices a researcher was hoping to expose and thereby end. (55)

But the ESRC understands this (somewhat), noting,

Much social science research has a critical role to play in exploring and questioning social, cultural and economic structures and processes (for example relating to patterns of power and social inequality), and institutional dynamics and regimes that disadvantage some social groups over others, intentionally or not. Such research results may have a negative impact on some of the research subjects. (22)

Likewise, Dyer and Demeritt write,

there are times when safety, either of the investigator or of research subjects themselves, means that research subjects cannot be informed about the true nature of research [such as] studies of human trafficking, illegal workers’ gang masters, and so on. In such cases the notion of asking participants to sign a consent form of the sort envisioned by the ESRC (2005) is ridiculous, and the ill of deception balanced by the importance of the research and a commitment to protecting anonymity. (57)

But the Research Ethics Framework recognizes this problem, stating, "informed consent may be impracticable or meaningless in some research, such as research on crowd behaviour, or may be contrary to the research design, as is often the case in psychological experiments where consent would compromise the objective of the research. In some circumstances – such as users of illegal drugs – written consent might also create unnecessary risks for research subjects." (21)

The real problem, it seems, is not the ethical content of the Research Ethics Framework, but the structure it employs to promote those ethics. Project-by-project committee review may be unable to handle the complexity and unpredictablity of social science research. As Dyer and Demeritt write,

Whereas drugs trials involving vast sums of money, or biomedical research on extremely vulnerable people in enormous pain need only gain anticipatory approval, the ESRC argues that because ‘purposes, methods, and intended uses’ of qualitative research evolves as it proceeds, this kind of research should be required to seek REC approval on multiple occasions. Such a disproportionate response is itself unethical if it overburdens researchers such that worthwhile research does not get done. (57)

The underlying issue is that committee review is a process specifically tailored to experimental research. The more a researcher fits the experimental model, the more detailed her protocol will be, and the more amenable to prior review. It's the open-ended, qualitative researcher who has the low signal-to-noise ratio, and for whom ethics committee represent such a waste of time.

IRB Says Women Can't Be Photographed

The New York Times reports IRB interference with a study that combines medical and social research. (Gina Kolata, "Fitness Isn’t an Overnight Sensation," 21 January 2009).

Carl Foster, an exercise physiologist at the University of Wisconsin, La Crosse, was amused by ads for a popular piece of exercise equipment. Before-and-after photos showed pudgy men and women turned into athletes with ripped bodies of steel . . .

“We said: ‘Wait a minute. You can’t change yourself that much,’ ” Dr. Foster said. So he and his colleagues decided to experiment. Suppose they recruited sedentary people for a six-week exercise program. Would objective observers notice any changes in their bodies?

The plan was to photograph volunteers wearing skimpy bathing suits and then randomly assign them to one of three groups: cardiovascular exercise, weight lifting or control. Six weeks later, they would be photographed again.

Their heads would be blocked out of the photos, which would be shuffled. Then the subjects and judges would rate the body in each photo on a scale of 1 to 10, with 10 being spectacular.

The volunteers were men, age 18 to 40 (the university’s human-subjects review board looked askance at having women photographed and rated like that).

This is another case where Canada's TCPS (even the existing version) shows a more sophisticated undertsanding of research ethics than does the Belmont Report or the U.S. regulations. Whereas the Belmont authors were concerned about vulnerable populations being inappropriately targeted for medical studies, they did not think about the arbitrary exclusion of populations, or the stigmatization of competent adults as incompetent. By contrast, Article 5.2 of the 2005 version of the TCPS states that "women shall not automatically be excluded from research solely on the basis of sex or reproductive capacity."

I am unaware of any U.S. regulation or guidance that so specifically forbids the kind of sexism displayed by the La Crosse IRB. Still, it's disappointing that that IRB thinks adult women incapable deciding for themselves whether to participate in a study open to men.

For this study's implications for advertising law, see Rebecca Tushnet's 43(B)log.

Canada Considers a New TCPS

The Research Ethics Blog alerted me to the relase of the Draft 2nd Edition of the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS), Canada's manifesto of research ethics. The document is strong in its recognition of critical inquiry. But it is weak in establishing mechanisms to meet its stated goals of proportionate review and reasoned decision-making. It fails to address some of the key concerns in the 2004 report, Giving Voice to the Spectrum, and threatens to leave social scientists not much better off than they are today.

Valuing Critical Inquiry

The new draft recognizes differences among the various activities that go by the name "research." Like the Belmont Report, it offers three core principles: concern for welfare, respect for autonomy, and respect for the equal moral status of all humans. Unlike Belmont (and particularly unlike 45 CFR 46), it recognizes that

these principles are not absolute. They may, at times, conflict. They do not apply in all circumstances, to all types of research, as is set out in the following chapters. How they apply and the weight to be accorded to each one will depend on the nature and context of the research being undertaken. (2)

Such insight is elaborated in sections on social research. In particular, the draft Statement embraces "critical inquiry," in passages longer and stronger than those in the existing Statement:

Research based on critical inquiry – focusing, for example, on public policy issues, modern history, or literary or artistic criticism – may involve interaction with living individuals, notably through interviews. Where the aim of the researchers is to engage in a critical examination of a body of artistic work, a public policy, other comparable types of work, the role of the REB should be limited to ensuring that researchers conduct their work respecting the professional standards of their discipline(s) or field(s) of research. The need to ensure freedom of inquiry and to protect the ability of researchers to criticize the work (or organization, political party, corporate enterprise, etc.) they are examining takes precedence over the need to protect individual parties from harm. (9)

Alas, the authors of the draft are inconsistent in their understanding of the role of critical inquiry. They write,

In certain areas of research in the social sciences and humanities, such as political science, economics or modern history (including biographies) . . . the purpose of the research may be to cast a critical eye on organizations, political institutions, or systems or individuals in public life. The outcome of these types of research may harm the reputation of public figures or institutions in politics, business, labour, the arts, or other walks of life. Such harm may, however, be an unavoidable outcome of research that seeks to shed light on or to critically assess the work of a public figure or institution. Where the purpose of the research is to advance knowledge about the workings, for example, of a public office or a public figure, the risk–benefit analysis by the REB should focus on whether the approach they have adopted respects the professional standards of the researcher’s discipline or fields of research. Just as a bruise is an unavoidable risk of research that requires a needle-stick, so harm to reputation is an unavoidable risk of certain types of social science inquiry, and it must be treated as such. (14)

That's fine until the last sentence, which suggests that the authors don't understand critical inquiry at all. A bruise is an unfortunate byproduct of a needle-stick. A harm to reputation is often the deliberate objective of critical inquiry. The trick, William Potter, is not minding that it hurts.

Seeking Proportionality

The draft Statement concedes that too much review can be a bad thing, stating that

The scope and intensity of ethics review should be proportionate to the level of risk involved. When those involved in the review of research tailor their level of scrutiny to the level of risk, they reduce unnecessary impediments and facilitate the progress of worthwhile and ethical research. This is the crux of proportionality, and it is a message that recurs throughout this Policy.

It is equally important that ethics review be appropriate to the disciplines, fields of research and methodologies of the research being reviewed. This means that REBs must understand the discipline and methodology under review and be able to assess the research on its own terms. (5)

To give researchers a fair shot, the Statement insists that "at least two members should have the relevant knowledge and expertise to understand the content area and methodology of the proposed or ongoing research, and to assess the risks and benefits that may be associated with the research," and that it seek ad hoc advisors when necessary. (58) It even requires "an established mechanism and procedure in place for entertaining appeals" (69), even permitting ad hoc appeals board, something forbidden by the existing statement.

Unfortunately, the draft does little to ensure "a proportionate approach to ethics review" or even to consider what that means. There's nothing in the statement to suggest that ethics boards are uniquely suited to uphold ethical standards, or that they are necessary for all kinds of human subjects research. So why must all research pass through an REB? While the idea of proportionality recurs throughout the document, when we get to its actual application (p. 63) we find only two levels of review: delegated REB review of minimal-risk research, and full REB review of everything else. A two-speed transmission is not very proportionate.

Giving Voice to the Spectrum called for "different approaches to ethics review that would allow REB blanket approval of programs of research based on the overall ethics strategy of the researcher (or team of researchers), within specified parameters" as well as "exemptions from review for social science and humanities research that involves standard practice in the discipline involved." (6)

Along these lines, a more proportionate system would allow for several levels of review, e.g.,

1. No review--for activities even a middle-schooler can do (e.g., conversations with family members)
2. Researcher certification--for activities where the key thing is to be sure the researcher is familiar with professional standards. The draft itself suggests that this is appropriate for much interview research.
3. Departmental committee review.
4. Delegated review.
5. Full board review.
6. Full board review plus outside consultation--for the riskiest research.

Another problem with proportionality is its dependence on the threshold of "minimal risk," which doesn't map well onto the type of critical inquiry that the policy seems to endorse. As noted above, the draft states that "where the aim of the researchers is to engage in a critical examination . . . the role of the REB should be limited to ensuring that researchers conduct their work respecting the professional standards of their discipline(s) or field(s) of research." That would seem to imply that the intent of the research, as well as its level of risk, should determine the level of review.

Ignoring Evidence?

Another big concern is that the document makes no requirement that REBs base their decisions on empirical reality. Giving Voice to the Spectrum called for "a shift in onus where, in order to require changes to a research proposal, an REB would be obliged to explain what identifiable harm has not been addressed, and how their proposed solution will ameliorate the problem." (6) That has not happened.

Article 6.13 of the draft provides that "the research ethics board shall function impartially, provide a fair hearing to those involved and provide reasoned and appropriately documented opinions and decisions." But what does "appropriately documented" mean here?

The 2005 TCPS also required "reasoned and well-documented decisions" (Article 1.9). But much conflict between REBs and researchers takes place when they cannot agree on what is meant by reasoned and documented. For example, when Tony Porter wanted to do some interviews about the governance of research ethics, his REB demanded that it be allowed to scrutinize the questions in advance, looking for some mysterious risk. Would such behavior count as a fair hearing with reasoned and appropriately documented opinions and decisions? Are boards empowered to dream up fantastic risks? Or should the standard be higher, demanding that ethics boards operate according to scholarly standards, and document the risks that they attribute to a project?

Again, compared to the Belmont Report, the ethics sections of this new TCPS are relatively nuanced, reflecting long thought and the input of a great many scholars. I'd like to see American universities list the TCPS as the "statement of principles" on their Federalwide Assurances. (See The Dormant Right to Ethical Self-Determination".) But by assuming the need for board review, and by failing to set standards for board decision-making, the draft leaves research vulnerable to the "unnecessary impediments" it seeks to eliminate.

OHRP to Revise Website

OHRP has called for feedback on the "content, format, navigation or any other aspect" of its website. Below are the comments I sent in.

---

Dear OHRP,

Thank you for your invitation for suggested improvements to the OHRP website. Before making any suggestions, I must first thank you for the existing website, which contains a great deal of extremely valuable information. As a historian, I particularly appreciate the historical material, including the Belmont Report oral history section, and the compilation of key notices from the Federal Register dating back to 1973.

With this in mind, let me suggest some additional material that would be helpful.

1. Responses to calls for comments

When OHRP issues calls for comments in the Federal Register, it routinely advises that "comments received within the comment period, including any personal information provided, will be made available to the public upon request." While I am grateful to OHRP staff for sending me electronic versions of these comments in replies to my requests, I suggest that all comments be posted on the OHRP website as well. The comments I have seen constitute thoughtful, informed positions on the regulation of human subjects research and mark significant contributions to the debate on the subject. They have been the subject of reporting in the press, suggesting the wide interest in them.

Placing the comments on the website would insure equal access to those comments by all concerned, and would solve the problem of electronic files that are too big for easy transmission by e-mail. Posting would also preserve the comments for future readers, a key concern given that many issues crop up repeatedly. For example, in 2007, OHRP requested comments on expedited review. Anyone responding to that call would have benefited from access to the comments received in reply to the similar call of 1997. But those earlier comments seem to have been lost to history.

2. Reports to which OHRP has contributed

In June 2008, the Human Subjects Research Subcommittee published a report called "Expedited Review of Social and Behavioral Research Activities." Perhaps this is not technically an OHRP report, but the Subcommittee was co-chaired by the acting director of OHRP, and both he and Mr. Drew of OHRP were members of the working group that produced the report. Yet there is no link to the report on the OHRP website, nor did I receive notice of it from the OHRP-L@LIST.NIH.GOV mailing list. When OHRP staff, acting in their official capacities, contribute to a report like this, it should be announced on the list and a link added to the OHRP site.

3. A complete copy of the Bell Report

Your current page, http://www.hhs.gov/ohrp/related.html, includes a copy of "Evaluation of NIH Implementation of Section 491 of the PHS Act, Mandating a Program of Protection for Research Subjects," published in 1998. But it does not include the technical appendices mentioned on page ii of that report, nor have I been able to locate a copy of those appendices. Posting them would be of value.

4. A link to OPRR Reports

The section on "Historical Documents" on the page http://www.hhs.gov/ohrp/related.html does not mention the OPRR Reports posted at http://www.hhs.gov/ohrp/dearcoll.htm. As the latter page indicates, most of these documents exist "solely as a reference for historical purposes." I am very glad that these documents are online, but I suggest that as historical documents, they deserve a link under the "historical documents" section. For that matter, I'd like to see the "Historical Documents/Information" section, including the Federal Register notices, given its own page, but I realize this wish may reflect my unusual perspective as a historian. I would also like a note added clarifying if the OPRR Reports now online represent all or just some of the OPRR Reports sent out during that office's existence.

Thank you again for this opportunity to comment on the website. If I think of additional suggestions, I will be sure to send them in.

Sincerely,

Zachary M. Schrag
Assistant Professor of History, George Mason University

Happy New Year, OHRP

In February 2007, the New York Times reported on scholars' frustration with IRB review of social research. OHRP assured reporter Patricia Cohen that it was working on the problem:

Bernard A. Schwetz, director of the federal Office for Human Research Protections, which administers the regulations, acknowledges that the guidelines covering the boards’ actions have not been clear enough and says he intends to make public new proposed guidelines before the end of the year.

Later in the article, Schwetz "said the new guidelines 'will give a lot of examples and will give more guidance on how to make the decision on what is research and what is not.'" This makes it clear that the 2007 call for comments on expedited review is not what he had in mind.

With the end of 2008, the promised guidelines are more than a year late.

This may be for the best. Schwetz seemed determined to continue OHRP's history of trying to regulate everything, while under Menikoff, we might expect better results. But OHRP and its predecessors have made a lot of empty promises over the years, and it's frustrating to see that tradition continued.