Sunday, October 23, 2011

In ANPRM Comments, Anthropologists Champion Free Inquiry

The American Anthropological Association has posted comments on the ANPRM to the website. Composed by AAA Committee on Ethics member Lise Dobrin and former Committee chair Rena Lederman, the 23-page document is stunningly eloquent and thorough in its indictment of the present IRB system and the shortcomings of the ANPRM. And it offers two bold and constructive suggestions that could point the way to a true reform.

The comment is organized in the haphazardly ordered sequence of the ANPRM's questions. Here, I extract some major features.

Critiques of the Status Quo

Taken together, the comment's descriptions of the status quo present a bleak picture.

  • "Even though their modes of planning differ from the experimental designs that IRBs expect, folklorists, interpretive sociocultural anthropologists and the like consider what they do to be research (in their respective disciplinary senses), present it as such to their IRBs, and are therefore advised by IRBs to submit applications. However, when they do so, their protocols are evaluated as 'poorly designed': as bad research, rather than simply not 'research' (in the present regulatory sense)."
  • "Anthropologists preparing to engage in participant observation routinely experience review of their protocols by board members who lack expertise in the standard methods anthropologists employ . . . This situation has demoralized individual researchers and encouraged widespread cynicism about the value of regulatory oversight."
  • "Our colleagues have reported that many rounds of requests for clarifications and changes can be as discouraging as outright disapproval."
  • "While we were preparing these comments, one anthropologist mentioned to us that she was forced to model her consent documents for classroom observations and interviews on forms designed for medical interventions such as intubation. Her IRB prevented her from adjusting her consent forms—using language that would be clear and pragmatically appropriate for the study population—to facilitate truly informed consent. We must recognize that foreign participants can be baffled and overwhelmed by procedures that Americans take for granted, such as 'initialing' a line."

Attacks on Muddled Language

The comment is particularly effective in its insistence that "the vagueness of the key regulatory definitions is a key cause of the relentless expansion in the workload of IRBs." It offers several examples of unhelpfully vague language, both in the current regulations and the ANPRM:

  • "The current definition of 'minimal risk' (45 CFR 46.102(i)) is ambiguous and the examples provided are not (and cannot be) comprehensively helpful."
  • "'Psychological' risk is a slippery, inherently subjective concept. As such, it is likely to become a fresh source of uncertainty and cautious reviewing, thereby undermining the point of this proposed rulemaking."
  • "Our expertise in cross-cultural research further leads us to view as fruitless efforts to specify across-the-board 'types of questions' or topics (e.g., sexuality) as inherently 'greater than minimal risk'. . . the ANPRM's effort to predetermine offensive or risky topics is inherently flawed."
  • "The term 'related methodologies' is inadequate: the proposed rulemaking offers no principled approach to creating that grouping or interpreting that phrase."
  • "[Question 23] asks about the circumstances under which it should be permissible to waive consent for research involving 'existing data and biospecimens.' We find the phrasing of this question extremely troubling as it reveals a lack of distinction between the very general concept of 'data' with the very specific notion of 'biospecimens'. So long as the framing of ethical regulations continues to take for granted a model of research in which 'biospecimens' are the prototypical form of 'data', there is no hope of achieving a reasonable, efficacious regulatory system."
  • "If our concern is to identify 'risks' to human research participants, then generalizable/not- generalizable is not a useful diagnostic for distinguishing reviewable from not-reviewable research. Literary scholars and historians write biographies that transform private individuals into public figures and shape their reputations (not infrequently for the worse) whereas survey researchers typically anonymize their respondents since their research questions concern social trends and mass phenomena. Nevertheless, survey researchers, but not biographers, have been made subject to the Common Rule. Clearly, 'generalizability' has not proved a helpful resource for comparing the relative risks of these activities."
  • "Existing lists include activities familiar to marketing researchers (e.g., taste-testing) but regularly omit mention of activities familiar to linguists, anthropologists, and others (e.g., conversation analysis, elicitation of grammaticality judgments, participant observation). The 'human subject protection' rationale for these omissions is obscure: one suspects that they may simply be unfamiliar to the list authors.
    Second, such lists are confusing to researchers and IRBs because different disciplines use methodological terms (e.g., 'interview') differently. Additionally, as the ANPRM itself notes, because research methodologies are dynamic, any enumeration of techniques is likely to need endless adjustment. The ANPRM indirectly acknowledges this problem when it ends its lists with the phrase 'and [other] similar procedures.'"
  • "The definition of 'human subject' makes quite explicit reference to biomedicine ('venipuncture' and 'medical records'). Indeed, the term is drawn from biomedical and experimental science usage, where 'subjects' are understood to respond to stimuli introduced and controlled by experimenters. However, most social and humanities research construes human participants as more proactively engaged (in various particular senses) in the research process: as open-ended interview 'respondents', oral history 'narrators', or ethnographic 'informants', 'consultants', and 'interlocutors'. Consequently, 'human subject' imports misleading connotations into IRB evaluations of social and humanistic research (introducing inaccurate study design expectations concerning 'subject selection' and the like). "

Skepticism of HIPAA

The comment emphasizes that "the use of HIPAA standards—'which were designed for dealing with health information'—are most definitely not 'appropriate for use in all types of research studies, including social and behavioral research.'"

A Radical Redefinition

The comment's boldest proposal is to redefine the scope of the regulation:

Whereas the Common Rule currently applies to 'research; involving 'human subjects' (45 CFR 46.102d, f), we recommend that a revised Common Rule apply only to the following two kinds of work:

1. Biomedical and other study procedures involving risks of physical harm to human participants: that is, more specifically, harm defined in 76 FR 44515 II(A) as "characterized by short term or long term damage to the body such as pain, bruising, infection, worsening current disease states, long-term symptoms, or even death."

2. Human experimentation and other methodologies whose results depend for their validity on limiting or controlling the information available to research subjects: that is, study designs reliant either on the passive withholding of information concerning what the study is about or on the active provision of misinformation: e.g., the use of placebos in biomedical clinical trials; the use of confederates in behavioral research concerning competition, conformity and the like; and the deceptive presentation of fictional narratives as actual news reports in social research concerning public opinion."
The comment later clarifies that "we recommend adopting a more precise definition of what is being regulated (human experimentation and study procedures involving physical risks to participants). Doing so would obviate the need for listing 'Excused' activities in the first place."

On the one hand, this would be a radical reduction in IRB authority; IRBs would be left with no power over researchers using surveys, interviews, and observation, power they have claimed since the first policies of the 1960s.

On the other hand, this is a very conservative proposal, in that it would restrict the regulatory power to the statutes on which it claims to be based.

The comment notes that it is proposing a "default-out reviewing strategy," in which activities could proceed unless they hit one of the triggers in the new definition, as opposed to the "current strategy [which] defines the object of IRB review relatively generally and then labors to identify and enumerate specific methodologies or types of study eligible for 'exemption' or 'expediting.'" Indeed. In 1981, HHS explicitly rejected the "suggestion that the regulations should define what is covered rather than list specific exemptions if research were exempted from coverage unless it met the criteria proposed by the commentators" lest "other categories of research involving significant risk . . . would be inadvertently exempted from coverage." This was a terrible mistake, and the AAA rightly condemns it.

The comment particularly wants participant observation, "sociocultural anthropology's distinctive research method," excused from IRB review.

I was puzzled by the combination of the comment's call to narrow the regulatory definition of research and its apparent expectation that there would still be some regulatory oversight over some social research. Thus, the response to question 15 imagines "obviat[ing] the need for listing 'Excused' activities in the first place," while the response to question 19 "strongly support[s] the idea that researchers who are engaged in 'excused' research adhere to a 'brief (i.e. no more than one week) waiting period' before commencing with their project."

I wonder if this apparent contradiction is the result of the ANPRM's format, which more or less forces respondents to argue in the alternative.

A Proposal for a Social and Humanistic Commission

The comment's second major suggestion is

the creation of a commission constituted specifically of social scientists (e.g., sociologists and the like), humanistic social researchers (e.g., cultural anthropologists and the like), and humanists (e.g., historians, legal scholars, and the like). Rather than adapting strategies developed to protect biomedical information--which are fundamentally incompatible with core intellectual and ethical commitments of humanistic social studies--this commission would be tasked with developing alternative guidance appropriate for their fields.
As the anthropologists note, such a commission is already functioning in Canada, where the Social Sciences and Humanities Research Council helps shape the Tri-Council Policy Statement and its interpretations. Like the call for a meaningful definition of research, this proposal is a bold assertion of what should be common sense. The anthropologists suggest the Association for Practical and Professional Ethics and the Consortium of Social Science Organizations as organizations that should have a greater role in shaping rules.

Again, I see some tension in this proposal between the comment's wish for more expert review and its defense of "the existing local model of IRB process." The comment rightly "attribute[s] the phenomenon of 'mission creep' in some part to the fact that institutions labor under the anxiety of federal audits. This contributes to an adversarial atmosphere in which it is only rational for boards and IRB administrators to emphasize bureaucratic documentation over situationally appropriate ethical decision-making." Yet its complaints about "variable interpretations by IRBs" could hint at problems with the local model itself.

Similarly, in response to question 9, the comment states that "We view a 'systematic, empirical assessment of the levels of risk' as impractical." Yet in just the previous section, a reply to question 8, the comment cites a National Academy of Science report to argue that "every exposure to [ionizing] radiation produces a corresponding increase in cancer risk." So there's an example of a practical, empirical assessment of the level of risk.

I am the last person to suggest an uncritical mapping of such a medical finding onto non-medical fields, but I still think some empirical evidence about nonmedical research can help guide rulemakers and local IRBs. For example, it seems that experience has shown that people who study universities should not count on being able to disguise the identity of their research sites (especially when they study their home institutions), and that interviews with survivors of trauma pose little risk of causing further trauma.

Respect for Historians

Though naturally emphasizing the concerns of anthropologists, the AAA comment goes out of its way to stress both the value of historical research and the dangers the ANPRM's muddier ideas pose. For example,

Taken literally, the wording of this question is directly applicable to historiography, which accords special credibility to the study of records collected (often by interactions with persons) for purposes unrelated to that of the investigator. To require consent in such a case 'based on the likelihood of identifying the research subject' would cause IRB (or HIPAA Privacy Board) workloads to rise dramatically and historical scholarship to grind to a halt."
I expect historians to make this point, but for anthropologists to do so is an act of collegial generosity.

An Endorsement of Freedom

In several sections, the AAA raises a value often missing in discussions of human subjects regulation: "the rights of citizens in a free society, researchers and participants alike."

My favorite expression of this value--and my favorite passage of all in the comment--comes in response to question 27, which asks about the risks that IRBs should consider. The anthropologists respond:

The research world is rife with projects whose results bore, interest, annoy, please, anger, or enlighten research participants (just as they do fellow researchers). This world is not generically the people's enemy. On the contrary, those of us working in US colleges, universities, news media, and research institutions have inherited traditions of free inquiry whose continuation is vital to this country's political, economic and social life. It would be deeply ironic if a regulatory system put in place to protect human beings were transformed into a device focused on restricting their power to know the world.
I have in the past been quite critical of anthropologists' approach to the IRB debate, in large part because of what I understood as their failure to accept that safeguarding freedom will inevitably bore, annoy, anger or otherwise harm some people being studied. The AAA's response to the ANPRM changes that. It strikes me as not only a much-needed set of comments on the proposals immediately before us, but also as a welcome assertion of anthropologists' rights and responsibilities in the pursuit of knowledge.

No comments: