Monday, March 19, 2007

PRIM&R Plans SBER Conference

Public Responsibility in Medicine and Research (PRIM&R), the professional organization for human subjects enforcers, has scheduled the "2007 Social, Behavioral, Educational Research (SBER) Conference: Sharing Tools and Joining Forces: Ethical and Regulatory Balance in SBER." The conference will be held in Broomfield, Colorado, on May 9 and 10.

The conference is notable because its planning committee includes two scholars who have written quite critically of IRB review of non-biomedical research: C. Kristina Gunsalus and Joan E. Sieber. (An earlier announcement also listed Felice J. Levine, but her name does not appear on the website.)

The conference program includes two sessions that promise to wrestle with the murky questions of definitions and exemptions:

A4. Developing Guidance on the Definition of Human Subjects Research (IRB Tool Kit I Track) [Please note that this is a double session and will end at 1:15 PM. This session has been designed with the dual purpose of discussing strategies and contributing to a written document that will provide guidance, definitions, and examples. This document will be electronically distributed to all Conference attendees following the meeting.]

A5. Developing Guidance on Applying the Exemptions
(IRB Tool Kit II Track) [Please note that this is a double session and will end at 1:15 PM. This session has been designed with the dual purpose of discussing strategies and contributing to a written document that will provide guidance, definitions, and examples. This document will be electronically distributed to all Conference attendees following the meeting.]

Potentially these documents could provide IRBs the guidance and cover they need to exempt survey, interview, and observation research with autonomous adults. While I am sorry I will not be able to attend the conference, I am very interested to see what it produces.

Sunday, March 18, 2007

My Problem with Anthropologists

(In honor of the discussion going on at Savage Minds, I present some thoughts on the role of anthropologists in the IRB debates.)
In terms of methods, anthropology and oral history seem to have a lot in common. Researchers in both disciplines enjoy learning about other people’s lives by talking to those people, often with a recording device. But the two fields have different ethical approaches, and I sometimes fear this makes anthropologists unreliable allies in the struggle for the freedom of research.
The problem starts with the code of ethics of the American Anthropological Association, which states that

anthropological researchers have primary ethical obligations to the people, species, and materials they study and to the people with whom they work. These obligations can supersede the goal of seeking new knowledge, and can lead to decisions not to undertake or to discontinue a research project when the primary obligation conflicts with other responsibilities, such as those owed to sponsors or clients. These ethical obligations include to avoid harm or wrong, understanding that the development of knowledge can lead to change which may be positive or negative for the people or animals worked with or studied.

Maybe I am missing something, but I don't see anything in the American Sociological Association’s code of ethics, the Principles and Standards of the Oral History Association or the American Historical Association’s Statement on Standards of Professional Conduct that obliges members of those organizations to avoid harm or to abandon the pursuit of knowledge lest someone be hurt. In seeking a balance between truth and inoffensiveness, the anthropologists have gone much further toward physicians’ Hippocratic standard of doing no harm than have their fellow social scientists. This decision may explain some troubling behavior:

1. Knee-Jerk Anonymity

My favorite anthropologist is Kathryn Marie Dudley, author of Debt and Dispossession and The End of the Line. Both books show how Americans struggle to reconcile their faith in the free market with their often conflicting belief that hard work should be rewarded regardless of market demand. That tension is central to many of today’s cultural and political debates, and Dudley did a magnificent job getting Midwestern farmers, teachers, and automobile workers to talk about their beliefs.
My complaint is that having done so, she fabricated names for her narrators, and barely felt the need to explain that decision. (Each book has a two-sentence note declaring that she wished to protect the privacy or confidentiality of the narrators, not why she thought this necessary.)
This has terrible consequences. First, it prevents other researchers from learning more about the lives of the people she studies, the way Dudley’s advisor, Katherine Newman, did by following up on the lives of her informants from a previous book. (Or at least it is supposed to. When I assigned Debt and Dispossession, one of my undergraduates did a quick search of newspaper databases and identified Dudley’s pseudonymous “Star Prairie” and some of its inhabitants.) And second, it suggests that the people who are the subjects of her books are not real, important people the way that other figures in the books—Lee Iacocca and Jesse Jackson—are. Unlike Iacocca and Jackson, their backgrounds need not be explored, and their words need not have consequences.
Most significantly for this discussion, the assumption that anonymity should be the norm contributes to the idea that interviewing is a dirty, dangerous activity. Of course some narrators wish their names to be disguised, and under some circumstances that is appropriate. But to see what happens when anonymity is the exception, not the rule, compare Dudley’s books to historian Leon Fink’s Maya of Morganton, another wonderful study of work in contemporary America. Fink offered anonymity to all his subjects, but the only ones who chose it were powerful executives and lawyers, not ordinary workers. In his book, interviews are opportunities to be heard, not sources of shame.

2. Disciplinary Imperialism

A voice of moderation in the IRB debates it that of Kristine L. Fitch, Professor of Communication Studies at the University of Iowa. (Since she defines herself as an ethnographer, I am including her in this rant about anthropologists. Perhaps that is unfair, but keep reading before you decide.) Fitch has been through IRB fights on both sides. In the 1990s, she writes, “I saw firsthand the aspects of human subjects review that so frustrate social science researchers, particularly those in the qualitative/ethnographic domain: applications full of questions aimed at biomedical research, requirements to obtain written consent despite cultural barriers to doing so, board members who said, in so many words, that their goal was to put a stop to as much research as they could.” (“Ethical and Regulatory Issues,” noted below.)
Fitch joined the University of Iowa’s social and behavioral IRB as chair and developed training materials that focused on the challenges faced by social scientists, in contrast to the medically-oriented materials mandated by most IRBs. At Iowa, researchers have their choice between “a two-hour workshop [focused on social and behavioral research], or completion of the National Institutes of Health (NIH) web-based certification course.” (See Fitch, “Difficult Interactions between IRBs and Investigators: Applications and Solutions,” Journal of Applied Communication Research 33 [August 2005]: 269–276. Apparently social-science ethics require live teachers, while medical ethics can be done in multiple-choice.) To help researchers and IRBs beyond Iowa, Fitch helped develop a CD and online training course called “Ethical and Regulatory Issues in Ethnographic Human Subjects Research.”
It is encouraging to see some training materials built from the ground up for social scientists. Rather than clubbing researchers over the head with more stories of Tuskegee, the CD focuses specifically on challenges in social-science research, such as how to protect privacy when studying sensitive topics, like eating disorders and illegal drug use.
But the CD goes wrong when it lumps in other disciplines with anthropology:

An issue that frequently creates tension between ethnographic researchers and IRBs has to do with translation of the ethical principles outlined in the Belmont Report into interpretations of federal regulations governing human subjects research. Some disciplines, such as the American Anthropological Association, the Oral History Association, and others have established systems of ethical principles specific to the kinds of research most characteristic of their areas. Those ethical principles arise from particular disciplinary histories and have been crafted by respected members of those professions. As such, they are often defended as more relevant, appropriate, and in fact more stringent than the necessarily more distant philosophy of the Belmont Report. Part of the disputable territory between researchers and IRBs that becomes contentious, then, is the distinction between abstract ethical principles and the regulations that spell out particular definitions, distinctions and prohibitions. Although researchers and IRBs would probably agree on ethical principles interaction between them is usually limited to the application of regulations to particular procedures, wording of consent documents, and so forth.

Did you catch the sleight of hand? Because anthropologists “and IRBs would probably agree on ethical principles,” Fitch assumes that “researchers and IRBs would probably agree on ethical principles.” I, for one, do not, and I see nothing in the guidelines of the Oral History Association that conforms to the Belmont Report’s demands for beneficence or what it calls justice. (See “Ethical Training for Oral Historians.”)
Beyond this misperception, I think Fitch is simply na├»ve about the operations of IRBs. “Ethical and Regulatory Issues,” for example, states that “IRB chairs and board members often have seen firsthand the negative consequences of . . . unanticipated problems.” Talk to researchers, talk to IRB members, read the postings on IRB Forum, and I think you’ll see that IRBs generally make decisions based on guesswork and what other IRBs are doing (in Fitch’s terms “long and thoughtful discussion among several reasonable people”), not firsthand or scholarly knowledge of the consequences of poor protocol design. If IRBs were required to support each decision with real-life examples of comparable projects gone bad, we would have many fewer restrictions on research, and essentially none on oral history.
And then there’s her claim in “Difficult Interactions” that “university administrators have a stake in human subjects oversight being carried out effectively and should be open to addressing problems within their IRB system. If they are not, the Office of Human Research Protection (OHRP) can be notified of hypervigilant regulation on the part of a local IRB. They can sanction IRBs for over-interpretation or misapplication of regulations when there is evidence that such is the case.” If OHRP has ever sanctioned an IRB for hypervigilance, I would love to hear about it.

3. Submission to the IRB Regime

What really concerns me are anthropologists in government, and here I am thinking of Stuart Plattner. Plattner served for thirteen years as the human subjects specialist for the National Science Foundation, and he worked to moderate some of the claims of IRBs. For example, the NSF’s website, "Frequently Asked Questions and Vignettes: Interpreting the Common Rule for the Protection of Human Subjects Behavioral and Social Science Research," created under his watch, includes the clearest statement by a federal agency that the Common Rule does not apply to classroom projects.
But Plattner is too ready to apply anthropology’s delicate ethics to other fields. In his 2003 Anthropological Quarterly article, “Human Subjects Protection and Cultural Anthropology,” he complains of “biomedical hegemony,” that is, the imposition of biomedical ethics on other disciplines. Yet in the same article he promotes a sort of anthropological hegemony when he writes, “no one should ever be hurt just because they were involved in a research project, if at all possible.” That’s consistent with anthropologists’ ethical statements, but other disciplines are happy to bring malefactors to account.
Where Plattner really gets scary is in his more recent “Comment on IRB Regulation of Ethnographic Research” (American Ethnologist 33 [2006]: 525–528). There he writes,

The journalist has a mandate from society to document contemporary reality. It is expected that this may involve an exposure of wrongdoing. A reporter’s reason for getting information from a person is to establish what happened. Those who speak to reporters accept the potential for harm that publicity may bring. Social scientists have no such mandate; we document reality to explain it. Our audience is professional, and society gives us no protection in the First Amendment. Our reason for getting information from individuals is to help us explain general processes. A normal condition for an ethnographic encounter or interview is that the information will never be used to harm the respondent.

The line that “our audience is professional” is simply defeatist; my undergraduates enjoyed Dudley’s books, and I hope plenty of readers outside the academy have found them as well. The bit about seeking to “explain general processes” is at odds with historians’ study of contingency, and I hope that other social scientists would reject that notion as well. But for the issue at hand, the key statement is that “society gives us no protection in the First Amendment.”
As a matter of jurisprudence, this is simply false. First Amendment liberties are common to all Americans, and if anything scholars enjoy heightened protection. In the 1967 case Keyishian v. Board of Regents of the University of the State of New York (385 U.S. 589) the Supreme Court held that “our Nation is deeply committed to safeguarding academic freedom, which is of transcendent value to all of us, and not merely to the teachers concerned. That freedom is therefore a special concern of the First Amendment, which does not tolerate laws that cast a pall of orthodoxy over the classroom.” It’s shocking that Plattner, who served so long as perhaps the most senior social scientist involved in shaping federal human subjects policy, has such little understanding of the law and such little concern for academic freedom.
Of course many anthropologists have written critically of IRB interference with their research. In the same journal that features Plattner’s dismissal of academic freedom, Richard Schweder eloquently argues that “a great university will do things that are upsetting,” citing Socrates, rather than Hippocrates, as the best Greek model for social science. (“Protecting Human Subjects and Preserving Academic Freedom: Prospects at the University of Chicago,” American Ethnologist 33 [2006]: 507–518). Yet who best represents the discipline: Schweder or Plattner? How far a leap is it from the American Anthropological Association’s subordination of the search for knowledge to Plattner’s suggestion that the First Amendment does not apply to scholarly research? Can anthropologists fight for academic freedom while holding that research shouldn’t hurt?

Tuesday, March 13, 2007

Why IRBs Are Not Peer Review: A Reply to E. Taylor Atkins

In his recent article on “Oral History and IRBs,” E. Taylor Atkins applauds the argument of J. Michael Oakes “that IRBs should be viewed as—and should behave as—peer review systems rather than as unaccountable ‘leviathans’ exercising authority from on high.” But Oakes and Atkins overlook profound differences between IRB review and peer review. The two systems are not merely different; they are antithetical.
The most obvious difference between peer review and IRB review is that peer review means review by peers—fellow scholars in one’s field. This was also the system envisioned in 1966 when the Public Health Service first mandated “prior review by [a researcher’s] institutional associates” before that researcher could received PHS funds, and to some degree it is the way IRB review works in biomedical research, where teams of physicians and other biomedical researchers review protocols by others in their field. But fundamental to the complaints of oral historians and others in the humanities and social sciences is the fact that IRBs are never (to my knowledge) composed mostly of scholars who conduct qualitative interviews. Indeed, most IRBs probably lack a single such researcher.
Even when IRBs do include such scholars, they operate in a web of laws, regulations, and guidelines written by biomedical researchers and bioethicists without input from historians. Consider the involvement of historians in shaping the present system:

Although historians have had no chance to shape the framework within which IRBs operate, by law and by their responsibility to their institutions, IRBs cannot ignore this framework. So it is not peer review at all that they impose, but values foreign to the historical profession.
One result of this peerless system is that IRBs are rarely able to offer the kind of expert advice we seek in good peer review. I have reviewed many manuscripts myself, but only when editors sought me out among a national or international pool of scholars. As a result, every manuscript I have read concerned some aspect of urban planning, infrastructure, or transportation—areas to which I have devoted years of research—or was designed as a textbook for use in the kinds of courses I regularly teach. Yet, as I note in my posting, “In Search of Expertise,” IRBs rely not on such a broad pool, but on whomever happens to be on campus and willing to serve on the local IRB.
Atkins also ignores the power of researchers to shape the peer review of their work. Certainly a university researcher who never submitted anything for peer review would be unlikely to win promotion, but Atkins forgets the choice researchers enjoy in deciding which peer-reviewed journal or press will receive their manuscripts, and which will be alternatives should the first choice reject the proposal. Comparable choosiness in ethical review is derided as “IRB shopping,” and university IRBs retain their monopolies (See Jeffrey Brainard, “Federal Agency Decides Not to Regulate 'IRB Shopping,'” Chronicle of Higher Education, 18 January 2006). Moreover, scholars and editors are free to ignore the advice of peer reviewers in preparing a final publication; how fondly I recall my first editor’s telling me not to worry about the difficult theory one reviewer wanted me to incorporate! Most importantly, scholars are always free to publish in non-peer-reviewed forums, such as this blog. In contrast, IRBs insist on the power to decide when a project requires their review.
Atkins recently attended a PRIM&R conference and “was astounded by the persistent ignorance among IRB administrators and board members in attendance about the special needs of SBER and oral history.” He responded by blaming the victim: “If we refuse to teach IRBs, how can they learn what we need them to know?” What he refuses to acknowledge is that IRBs’ power frees them of any incentive to learn. A publisher whose peer-reviewers offered consistently bad advice would soon lack for material. An IRB can mistreat researchers yet suffer no loss of power; its decisions are final.
While peer review is a persuasive effort by volunteers, IRB review is a coercive practice by agents of the state. Leviathans or not, IRBs do little but exercise authority from on high, empowered by federal regulations stating that “an IRB shall review and have authority to approve, require modifications in (to secure approval), or disapprove all research activities covered by [federal] policy,” and that no university official can overturn such decisions. Institutions that disobey this mandate risk losing millions of dollars in funds; individual researchers risk denial of degrees, promotion, or even the right to continue any research.
Take away this coercive power, and perhaps historians will learn to respect IRBs. Until then, expect them to resist.

Savage Minds

Professor Rena Lederman is guest-blogging on Savage Minds, an anthropologists' blog. She plans to provoke discussion on the "special problems ethnographers face in negotiating with IRBs, especially given the expansion and intensification of IRB scrutiny over the past several years."

Sunday, March 11, 2007

AHA Perspectives: Forum on IRBs

The March 2007 issue of the American Historical Association's newsletter, Perspectives, features three essays on IRBs:
and I plan to comment on the Atkins essay in a separate posting. For the Shopes piece, I offer a small factual correction: 45 CFR 46 was first promulgated in 1974, not 1981. See Final Rule, HEW, Protection of Human Subjects, Federal Register, May 30, 1974. [PDF] I am working on a history of the regulation of the social sciences as "human subjects research" from 1966 to 1991, which I hope will clear up some of this chronology.