Friday, March 19, 2010

Twenty-Six Percent of Boxes Go Unchecked

Approximately 74 percent of U.S. institutions with federalwide assurances apply all or part of 45 CFR 46 to all research regardless of support, down from more than 90 percent of such institutions a decade or so ago, according to OHRP officials.

[Carol Weil, Lisa Rooney, Patrick McNeilly, Karena Cooper, Kristina Borror, and Paul Andreason, "OHRP Compliance Oversight Letters: An Update," IRB: Ethics & Human Research 32, no. 2 (2010): 1-6.]

The authors write that since their previous review of determination letters, published in 2003 and covering the period 1998-2002,

more institutions have decided not to extend their FWA to research not supported by HHS, which means that OHRP has jurisdiction over fewer research studies than in the past. Based on an informal review of a sample of institutions, approximately 74% of domestic institutions currently holding an FWA that formerly held a Multiple Project Assurance (MPA) apply either subpart A or subparts A, B, C, and D of 45 CFR 46 to all research regardless of support. On the other hand, greater than 90% of those same institutions applied 45 CFR 46 to all research regardless of support when they held MPAs.

Along with that finding, the authors "describe [their] review of 235 compliance oversight determination letters that the Office for Human Research Protections (OHRP) issued to 146 institutions between August 1, 2002, and August 31, 2007." Such a large dataset would seem to be a rich source of information for those interested in how researchers and institutions go wrong.

Unfortunately, determination letters are generally opaque documents that fail explain the events leading up to a complaint. For example, OHRP's March 2007 letter to New Mexico State University tells us a bit about office procedure at New Mexico State, but nothing about the design of a project called "The Impact of Education in Navajo Nation Border Community Public Schools on the Hearts, Minds, and Spirits of Navajo Students," or whether that project violated any research ethics.

Because of this opacity, a review of determination letters can tell us that 56 percent of institutions were cited for problems with the "IRB initial review process" while only 3 percent were cited for misapplication of exempt categories of research. But it can't tell us what percentage of citations concerned ethnographic fieldwork, what percentage involved graduate students, or what percentage involved questions of whether data were publicly available. Nor can it tell us whether any one of these 235 letters documented actual harm to a research participant.

(I am puzzled by the finding that OHRP issued 260 citations for deficient "IRB-approved informed consent documents/process" but only 19 for "Failure to obtain informed consent of subjects." Does that mean that deficient processes still work 93 percent of the time?)

OHRP is missing an important opportunity. When courts rule on cases, they take care to explain not only the legal principles involved, but also the underlying facts, so that other courts can understand which principles should be applied to certain fact patters. By failing to document the substantive concerns that led to its investigations, OHRP has failed to educate researchers and IRBs about what kinds of research are most likely to lead to trouble.

Monday, March 15, 2010

Sociologists Split on Ethics Review

Two researchers from the University of Toronto find Canadian qualitative sociologists divided in their responses to research ethics boards.

[Judith Taylor and Matthew Patterson, "Autonomy and Compliance: How Qualitative Sociologists Respond to Institutional Ethical Oversight," Qualitative Sociology, published online 23 February 2010, DOI: 10.1007/s11133-010-9148-y]

Three Ethics Orientations

Judith Taylor and Matthew Patterson interviewed one qualitative sociologist from each of 21 of Canada's 22 PhD-granting sociology departments. They found that respondents fell into three general groups of "ethics orientations," which they characterize as "active engagement, apparent accommodation, [and] overt opposition."

The angriest sociologists are "overt opposers" who "passionately defend unencumbered intellectual pursuit, sociological traditions of inquiry, experiments and innovations in inquiry, the primacy of sociological understandings of ethics, faculty discretion, and critical or autonomous sociology. Such scholars see institutional ethical review as putting intellectually-credible human-based sociology and academic freedom in peril, threatening the nature and legacy of the discipline of sociology and the job of sociologists." Faced with IRB/REB requirements, they do whatever they can to resist, even deceiving the boards when necessary.

At the other extreme, "active engagers believe regulators and practitioners can collectively manage conflicts by working together as virtual peers within the same committees and institutional settings." They welcome ethics review and may even serve on boards themselves.

In the middle, "apparent accommodators accept the necessity of ethics regulation, but fret over the practical implications of these regulations for their research and teaching." This is the largest group, comprising ten of the 21 responses, with six opposers and five engagers making up the rest. As the authors note, this split means that while three-quarters of sociologists think ethics review is necessary, two-thirds of those are also engaged in "some form of resistance toward ethical oversight."

Though Taylor and Patterson "argue that the ways in which academics understand and act toward ethical oversight is far more complex and variable than is represented in the current literature," the range of responses they report closely resembles the range of viewpoints I have reported on this blog and in my forthcoming book. For example, "Communication Scholars’ Narratives of IRB Experiences," Journal of Applied Communication Research 33 (August 2005): 204-230, are not broken down into categories, but they express views quite similar to those found here.

Where You Stand Depends on Where You Sit

More intriguing are Taylor and Patterson's explanations of why scholars think as they do. They attribute viewpoints to scholars' age and status, noting that "Overt opposition tends to be exhibited mostly by senior men in the discipline who began teaching in the 1970s or 1980s, although some senior women are also opposers. Overt opposers also tend to hold appointments at the most research-intensive and elite institutions." The more tolerant apparent accommodators "tend to be mid-career scholars at research-intensive institutions who share their oppositional colleagues’ analyses while also internalizing blame for the transgressions of their predecessors." And active engagers "tend to be younger, untenured women faculty with less than a decade of teaching experience, holding positions at less research-intensive universities."

Taylor and Patterson suggest that as the youngest cohort, the engagers will inherit sociology. "As the generation of opposers retires and engagers increase in prevalence and power, the range of available orientations and methodological approaches may markedly narrow, with possible effects on the production of sociological knowledge."

A Liberal Is a Conservative Who Has Been Arrested

Another possibility, however, is that attitudes toward ethics boards reflect knowledge gained from reading and experience. Direct experience with ethics boards breeds skepticism. Only one respondent is quoted as having direct knowledge of sensible advice dispensed by an ethics committee: the suggestion of a "graduated consent form." But many faced inappropriate demands by ethics boards, either regarding their own research or that of their students. As the authors note, when accommodators "sense that those reviewing their ethics protocols do not share [their] understanding of sociology, they lose faith in the ethics review process." One or two incidents of "decisive trespass" may be forgiven. But the reality of ethics review eventually may wear out their trust.

The only thing that keeps these frustrated accommodators from becoming opposers is their belief that "a history of ethical violation" by sociologists "prevents them from more outwardly opposing their boards." I would be curious to know just what the respondents believed about the history of their discipline, and how they had come to those beliefs. Do they get their history from historians, from sociologists, or from ethics boards?

If attitudes toward ethics boards are shaped by experience, rather than status, then opposition may not fade as today's opposers retire. Graduate students may enter the profession as idealistic engagers. But as they pursue their careers, the missteps of ethics committees (and perhaps a better grasp of history) could drive them into the camps of accommodation, then opposition. Today's engagers may be tomorrow's accommodators, and today's mid-career accommodators could be tomorrow's senior opposers.

Thursday, March 4, 2010

IRB Warns That Opinions May Vary

Michael Bugeja, director of the Greenlee School of Journalism and Communication at Iowa State University, reports a run-in with his IRB.

[Michael Bugeja, "Avatar Rape," Inside Higher Ed, 25 February 2010.]

Bugeja was interested in "avatar rape": forced, simulated sex in a virtual environment such as Second Life. As a journalism professor, he wanted to know what other university scholars and administrators thought about the problem. But his IRB imposed conditions that discouraged responses.

In researching the phenomenon, I sought viewpoints from directors of information technology and women's studies at Big XII and other peer institutions. My research assistant Sam Berbano and I spent two months working with our Institutional Review Board, seeking approval to post our survey online.

Given the sensitive nature of the topic, the IRB asked us to warn survey participants about possible harm to their reputations should their responses be published. To lessen risk, the IRB also required signed copies of consent to anyone responding to our survey. So we opted for a snail mail version with a disclaimer: "A risk of participation in this survey may arise if some may find your opinions in the free-response section at variance with their own."

My research assistant wondered how a survey measuring opinion about avatar rape could have more potential for harm than participation in a virtual environment in which such a digital act could occur.

As it turned out, only one respondent out of 43 provided comments for this essay.

Is variance of opinion the kind of risk to "reputation" against which 45 CFR 46 is supposed to protect? I don't think so, but who knows? The interagency group that inserted "or reputation" into the 1991 regulatory amendments never explained its decision, even in the face of an objection that "reputation is a subjective term that is difficult to define operationally."

What I can say is that as a scholar and educator, I strive to expose people to opinions they do not share. At Iowa State University, such an outcome is classified as a hazard.

Monday, March 1, 2010

A Few More Words on TCPS 2009

Ted Palys and John Lowman of Simon Fraser University kindly sent me a copy of their comments on the latest draft of the TCPS. The also alerted me to the Novel Tech Ethics site, which is publishing various comments on the draft.

Palys and Lowman find the latest draft to be an improvement over the December 2008 version, but they remain dissatisfied with elements of its treatment of social science research. Their main concern is that REBs will remain "dominated by members who have no specialized knowledge of the practices in a particular discipline can undermine the integrity of ethics review and, all too often, make it a process to be undertaken and survived instead of a discussion to be welcomed." To remedy this, they suggest that institutions be encouraged to establish multiple REBs with expertise in particular areas. They also make a number of suggestions for smaller reforms.

While Palys and Lowman have a number of significant concerns, it's worth noting that their comments on this draft are generally much warmer than their critique of the 2008 draft. This confirms my sense that while the latest TCPS has serious flaws, it is the best effort yet to include the concerns of social scientists into a general research ethics document.

Since today is the deadline for comments, I submitted an addendum to my February 6 comments:

To the Panel on Research Ethics,

On February 6 I submitted comments on the Revised Draft 2nd Edition of the TCPS (December 2009). I would like to amplify those comments slightly.

In my earlier comments, I noted my concern with Chapter 5's language warning about the possible stigmatization of groups. As I stated, I believe that this should not be the basis for restricting research in the social sciences and humanities.

Since then, I have reread the report, and I came across the following language in chapter 1:

The welfare of groups can also be affected by research. Groups may benefit from the knowledge gained from the research, but they may also suffer from stigmatization, discrimination or damage to reputation. Engagement during the design process with groups whose welfare may be affected by the research can help to clarify the potential impact of the research and indicate where any negative impact on welfare can be minimized. Researchers must also consider the risks and potential benefits of their research and the knowledge it might generate for the welfare of society as a whole. Where research on individuals may affect the welfare of a group(s), the weight given to the group’s welfare will depend on the nature of the research being undertaken and the individuals or group in question. This consideration does not imply, however, that the welfare of a group should be given priority over the welfare of individuals.

Again, let me emphasize that legitimate, critical research in the social sciences and humanities often is damaging to groups. As Edward Shils noted in 1973, "Any factual description, however objective and true, might be offensive to those who are sensitive and whose actions and qualities are such as to fall short of reasonable standards." I therefore do not consider the passage on group harms to reflect "core principles [that] transcend disciplinary boundaries and therefore, are relevant to the full range of research covered by this Policy."