Saturday, April 25, 2009

UMKC's Respectful Oral History Policy

The University of Missouri-Kansas City (UMKC) has posted a promising new policy: "Social Sciences IRB and Oral History."

The policy has a number of elements that set it apart from the typical university policy, which seeks to cram oral history into a system designed for medical experimentation. Instead, it adapts only those elements of the medical IRB system that encourage historians to follow their own discipline's ethics and best practices.

I suggest that readers of this blog read the whole policy, but here are some highlights:

1. Respect for Critical Inquiry

As I have written repeatedly on this blog, historians do not take the Hippocratic Oath, and should not promise not to harm the people they interview. Any IRB that imposes the Belmont Report on historians is asking them to forswear their own ethics.

UMKC understands this. Its policy notes that

akin to a journalist or lawyer, an historian is also responsible to a wider public to recover a shared past “as it really happened.” In keeping with the public role of an historian in a democratic society, these responsibilities, especially when conducting narrative interviews, can necessitate a confrontational style of critical inquiry. So while historians do not set out to hurt their interviewees, oral historians are expected to ask tough questions in their interrogation of the past.

2. Respect for Peer Review

The UMKC neither subjects oral historians to the whims of board members unfamiliar with their field, nor does it leave them on their own. Instead, it offers scholars a number of relevant readings, including publications of the Oral History Association, and then encourages them to talk to colleagues knowledgeable about interviewing:

After reviewing these resources on their own, the researcher is strongly encouraged to discuss their research protocol with peers before implementing their research protocol. In some cases, peer review by members of one’s own department would be most useful; in other cases, a researcher might be better served by seeking review from a colleague in a different department.

To foster these kinds of conversations among the faculty, the Social Sciences IRB Subcommittee for Oral History will hold two meetings per semester . . . to discuss “Best Practices” in oral history. Faculty experts in oral history will guide these conversations . . . These meetings are designed to meet the needs of researchers seeking advice and peer review for their research protocols. They are also designed to meet the needs of Chairs and/or designees interested in learning how to advise researchers in their departments to make responsible decisions regarding oral history.

3. Respect for OHRP's Pledge

UMKC takes seriously the carefully negotiated 2003 agreement between the American Historical Association and the Oral History Association and OHRP, even posting a copy on its website. The university elaborates on that agreement:

At UMKC, we draw a distinction between idiographic research that uses oral histories to describe the unique story of some particular social group or individual, which does not constitute “human subjects research”; and nomothetic research that employs oral histories in the hopes of contributing to a general theoretical or comparative debate about the human nature or behavior, which does fall under the category of “human subjects research."

While I confess that the terms idiographic and nomothetic are not in my working vocabulary, I believe they do express a real difference between the ethics of oral historians and those of other scholars. If one is interested in a general theoretical or comparative debate about the human nature or behavior--as many social scientists seem to be--then it makes less sense to single out individuals for potential honor or calumny. Writing about unique individuals or groups changes one's responsibility toward the individuals interviewed.

4. Respect for Researchers

Policies like UCLA's infantilize researchers, making them submit every judgment to an administrator. By contrast, UMKC trusts its scholars:

The bottom line is that the researcher makes these determinations in careful consultation with the Chair of the department or another official designee appropriate to the kind of study being planned. Together this determination is based on shared understanding of all relevant guidelines and their shared expertise in their specialized field of scholarship.

5. Respect for the IRB

Even as it empowers historians, the UMKC policy keeps the IRB involved, making it a resource, rather than an obstacle. Researchers still have to learn something about human subjects regulations, and they must complete a form explaining why they have determined that their policy does not fall under federal regulations.

(The form's demand for an explanation of "no more than 1500 characters" sounds suspiciously bureaucratic, but it's a good length for the presentation of a single idea--about the same as the 150-word limit for a New York Times letter to the editor.)

More importantly, the frequent meetings of the Social Sciences IRB Subcommittee on Oral History suggest that some scholars at UMKC have devoted their time to helping colleagues deal with the real ethical challenges of oral history.

The website explaining the policy notes that it was developed by "a group of faculty and administrators involved with the Social Science Institutional Review Board (SSIRB) . . . with input from members of the SSIRB, the College of Arts & Sciences, and the Faculty Senate at UMKC." I congratulate all the scholars and administrators who developed this innovative system, and I hope it works as well in practice as it reads on the screen.

With this policy, UMKC joins Amherst College, Columbia University, the University of Michigan-Ann Arbor, and the University of Nebraska-Lincoln Policy on a small but growing list of schools that have adopted OHRP's 2003 position removing most oral history research from IRB jurisdiction. Five schools not very many, but it's five more than the AHA could find in February 2006. Who will be number six?

Tuesday, April 21, 2009

Deregulation "Is Not Going to Happen"

Linda Shopes kindly alerts me to the April 20 issue of COSSA Washington Update, the newsletter of the Consortium of Social Science Organizations, which reports on an April 1 meeting of the National Academies’ Board on Behavioral, Cognitive, and Sensory Sciences, at which IRBs were discussed.

Here's the key passage:

Philip Rubin, CEO of Haskins Laboratories in New Haven, CT, and former director of the National Science Foundation’s (NSF) Division of Behavioral and Cognitive Sciences, chairs the Board. He began the session with a review highlighting the difficulties social/behavioral researchers have had with the current system under the Common Rule regulation and its interpretation by campus Institutional Review Boards (IRBs). Complaints have been loud, but mostly anecdotal . . . Once again the bottom line is that despite efforts by Joan Sieber and the Journal of Empirical Research on Human Ethics, which she edits, there are still large gaps in our empirical knowledge of how the system works for social and behavioral scientists.

Rubin was followed by Jerry Menikoff, new head of the U.S. government’s Office of Human Research Protections (OHRP). Menikoff announced that he was all for “flexibility” in the system and that “changes can be made.” He also endorsed conducting more research. He rejected the arguments of the American Association of University Professors and Philip Hamburger of Northwestern University Law School that IRBs violate researchers’ first amendment rights. He acknowledged the importance of expedited review, but stated quite clearly that “removing minimal risk research from the system is not going to happen.”

I don't want to make too much of these comments; an OHRP spokesperson tells me that they were an extemporaneous response to Rubin, and not prepared remarks. Still, I am disappointed. Menikoff's comments suggest a retreat from his earlier concession that "flexibility" often can be code for arbitrary power. And it's a pity for a public official to insist that a given policy "is not going to happen" even as he endorses more research. Wise governance depends on making policies after finding facts, not before.

Friday, April 17, 2009

Macquarie's Innovative Ethics Training

In previous posts and my 2007 essay, "Ethical Training for Oral Historians," I have complained about standardized, medicine-centric ethics training systems like the CITI Program and called for training programs better tailored to individual disciplines.

Lisa Wynn of Macquarie University (also known as MQ) has alerted me to just such a program she created with Paul H. Mason and Kristina Everett. The online module, Human Research Ethics for the Social Sciences and Humanities, has some elements that I find inappropriate. Overall, however, it is vastly superior to the CITI Program and comparable ethics programs I have seen, and it deserves attention and emulation.


Relevant Examples

The CITI Program's "History and Ethics" module offers five "Events in Social & Behavioral Research" as cautionary tales. Two concern psychological experiments (Milgram and Zimbardo), two concern decades-old observations using deliberate deception (Wichita Jury and Laud Humphreys) and the last is Francis Flynn's more recent, but also deliberately deceptive, study of restaurant responses to customer complaints. This choice of examples suggests that the only ethical dilemmas facing ethnographers concern deception, and that they do not occur frequently. The University of Iowa's Ethical and Regulatory Issues in Ethnographic Human Subjects Research is broader in its concerns, but offers few concrete examples.

The MQ program, by contrast, offers two old stories (Zimbardo and Humphreys), but it freshens them with stories of two recent controversies: the U.S. Army's recruitment of social scientists for its Human Terrain System initiative and Sudhir Venkatesh's doctoral research in underground economies in Chicago. Another case study appears in the section on "Research with Aboriginal and Torres Strait Islander Peoples," describing the collection of stories, songs, and artifacts by anthropologist Ted Strehlow. These collections have become the subject of considerable controversy, and the program offers both Strehlow's perspective and that of his critics. Beyond these case studies, the MQ program offers short vignettes in which ethnographers had to decide how to conduct research without hurting their informants or themselves. In some cases, the researchers are given pseudonyms; in others, real names. The authors even include accounts of dilemmas they themselves have faced.

The stories that make up the bulk of the MQ program are more relevant to today's ethnographers than hearing about what Laud Humphreys did forty years ago, or about recent medical research gone wrong. They show that ethical challenges are not confined to the distant past, but face social scientists today. And they broaden the challenges facing scholars to questions of confidentiality, government sponsorship, intellectual property, and other contemporary concerns.

Best of all, these stories emphasize real ethics, rather than the regulatory compliance at the heart of the typical program. When I went back to the CITI Program to write this entry, I was immediately confronted with a quiz question about whether a hypothetical foreign institution needed its own federalwide assurance for a hypothetical project. I'm sure this is very important to some IRB administrators, but it is absurd to demand that every researcher master such arcane requirements. The MQ program is not wholly free of this dross, but it is better than others I have seen.

Like other training programs, the MQ program is heavy on examples of what not to do. But it also offers something else: a positive example. One section describes how medical anthropologist Paul Farmer not only studied tuberculosis in Haiti, but also established a medical clinic there. As inspirational as this story is, I think this section could be expanded to offer stories of ethnographers who helped communities simply by doing good ethnography--not by providing medical care. Oral historians often get to work with people who are happy to have their words recorded for posterity. I would hope that sociologists and anthropologists have equally gratifying experiences. The MQ program does offer additional readings, which may contain these kinds of examples.

Room for Debate

The CITI Program lists "Events in Social & Behavioral Research" with attendant "ethical problems." The MQ program, by contrast, describes "ethics controversies" and "debates about the ethics," suggesting that there is no single right answer to the conundrums faced by scholars in the field.

To emphasize that point, the MQ program rejects the single-answer, multiple-choice quizzes that are the staple of the CITI Program. Instead, the MQ program offers questions like this:

Say you are doing research on cigarette smoking, but as you talk to the smokers, they start telling you about the illicit drugs they use. What do you do? (47)

There are no multiple choices here, just a stark presentation of an ethical challenge a scholar might reasonably face.

Elsewhere, the program does offer multiple choice quizzes, but not with the simplistic approach of the CITI Program. For example, one screen invites the user to choose one of four strategies for avoiding the revelation of confidential information. Whatever one chooses, the program replies that "any of the above are possible strategies for protecting your informants' identities, but some are better strategies than others," and then elaborates.

The program even admits that university ethics committees don't have all the answers, and sometimes have the wrong answers. It offers the example of Kristina Everett, one of the program's authors, who offended a longtime friend by confronting her with the written consent form demanded by her university. The program concludes,

Institutional ethics rules can . . . fall short of researcher’s own moral responsibilities and commitments to a particular cause. Ultimately, researchers must make their own decisions about what is ethical in the context of the particular research situation, in dialogue with their research participants. (103)

Scholarly Norms

The MQ program respects its students by adhering to two basic norms of scholarly writing.

The first is citation. Many of the CITI Program's statements are unattributed, and in some cases factually inaccurate. When a statement is attributed, the program often gives just a single source. The MQ program, by contrast, provides several readings for each of the case studies it presents, in some cases with alternative viewpoints.

The program also provides citations for statements throughout the text, adding authority. A warning that the U.S. government might seize a scholar's laptop at an airport might sound alarmist were it not for a link to a news article on the subject.

Another norm is that of open access. Many ethics training programs are open only to affiliates of a single university or, in the case of CITI, to affiliates of institutions that subscribe to the service. This robs scholars of the chance to compare and critique rival systems, to find areas of agreement and disagreement, to check facts, and to do all the other work that scholars usually do in their quest for truth and wisdom.

The MQ program requires registration, but that registration is free and open to all. Better still, the authors have licensed the text under a Creative Commons Attribution Non-Commercial Share Alike license, giving others the chance to "download, redistribute, remix, tweak, and build upon the text of this work non-commercially, as long as they credit the original authors and license their new creations under the identical terms." These two decisions offer the opportunity and the challenge for other institutions and disciplines to provide even better training.

The downside of such openness is that it leaves a training program open to criticism from cranky bloggers halfway around the planet. Let's move on to some of the program's weaknesses.

Areas for Improvement

Irrelevant History

The first section of the Macquarie program takes readers through about 15 screens concerning the history of abuses by medical researchers--Nazis, Tuskegee, and the more recent, dubious drug trials in Africa--along with descriptions of the various ethical codes designed to prevent recurrences. I hate this. It presents biomedicine as the archetype of research and social research as a deviation from that norm. And it's just bad pedagogy to start a course with material not important to the student.

If this is truly training for the social science and humanities, why not tell just the story of ethical debates in the social sciences and humanities? Or, if social scientists must know about Nuremburg and Tuskegee in order to communicate with university ethics officers, why not leave that to the end?

Disciplinary Bias

The MQ program is entitled "Human Research Ethics for the Social Sciences and Humanities" and it notes that "social science research -- including psychology experiments, quantitative surveys, oral history collection, and ethnographic research or participant observation -- raises its own peculiar problems for ethical research practice." (24) But the program does not give equal weight to the problems of each type of research or the ethics embraced by its practitioners.

The case studies include no examples of psychological experimentation since Zimbardo's prison study, or any survey or oral history research, or other fields not included in the list, such as geography, journalism, and creative writing. (Keep in mind that the Australian National Statement on Ethical
Conduct in Human Research
concedes that its broad definitions "could count poetry, painting and performing arts as research." [2:8]) Instead, almost all of its examples, and its quotations from ethics statements, come from anthropology.

Presenting case studies and ethical statements from all the fields in the social sciences and humanities would bloat the training program beyond usefulness. Rather than add more content, the program should strip references to survey and oral history research and adopt a more humble title: "Human Research Ethics for Ethnography."

Even this might be problematic, given the divergence in research ethics among various stripes of ethnographers. In particular, the program ignores debates over the principle of beneficence. In describing the controversy over the Pentagon's Human Terrain System (HTS), the authors write:

The reason for anthropological opposition to the Human Terrain System (HTS) lies in the discipline's code of ethics. Specifically, HTS opponents charge that Human Terrain Teams cannot guarantee several basic tenets of ethical research, including 'Do no harm,' the principle of informed consent, and freedom from coercion. (40)

Since when are these "basic tenets" of ethical research? For anthropology, the answer would seem to be only since the 1990s. [Carolyn Fluehr-Lobban, "Ethics and Professionalism in Anthropology: Tensions Between Its Academic and Applied Branches," Business and Professional Ethics 10 (1991): 57-68.] For other scholarly disciplines, the answer is not yet, particularly since the idea of "do no harm" would preclude investigative journalism and other critical inquiry. For many scholars, bringing harm to a malfeasor is highly ethical practice, but the MQ program seems to think that all scholars have embraced anthropology's gentleness.

I find this narrowness particularly disappointing because just last summer, Wynn herself raised the question of disciplinary differences in a post about HTS on Culture Matters. She understands that anthropologists are unusual in some of their ethical beliefs, and I wish that insight had made it into the MQ training.

(See "My Problem with Anthropologists.")

Incomplete Accounts

I have additional concerns about the treatment of two of the four social-science case studies in the program.

The section on Laud Humphreys is somewhat ahistorical, neglecting the ethics debates that were raging through sociology at the time Humphreys planned his work. I'm also perplexed by the statement that "Humphreys argued that the deception was justified, because there was no other way to obtain such information in a consented way because such sex practices were so highly stigmatized." (37)

Really? Here's what he wrote in 1975: "I am forced to agree with my critics regarding that part of my study in which I traced license numbers and interviewed respondents in their homes . . . I now think my reasoning was faulty and that my respondents were placed in greater danger than seemed plausible at the time.” [Laud Humphreys, Tearoom Trade: Impersonal Sex in Public Places, enlarged ed. (Chicago: Aldine, 1975), 230.]

As for the section on the Human Terrain System, it is based almost entirely on the writings of critics. For example, the program asks,"If the anthropologist has no control over what the military will do with the information, then how can an anthropologist fully explain to people the real risks involved in research?" Not only does this question ignore the fact that not all Human Terrain team members are anthropologists, it also assumes that those members do not control the information they gather. For a challenge to this assumption, see Adam Silverman, "The Why and How of Human Terrain Teams," Inside Higher Ed, 19 February 2009.

Awkward Layout

Writing this review was slowed considerably by the poor design of the MQ program's site. The program consists of 103 screens which must be viewed one at a time. Since one must often scroll down to reach the "next" link, it took hundreds of mouse actions to get through the site. With a big enough monitor, you can enlarge the window to include all the text and the navigation links without scrolling, but then you end up with line lengths of 100 or more characters, making the text hard to read.

If you stop clicking for a while, the site logs you out. And while you can return to the chapter you were on, you cannot navigate straight to the page. So if you pause to take notes, as I have while writing this entry, you may need several hundred mouse clicks to get through the whole program. I also found a noticeable pause after each screen, further delaying my reading. There is no way to print or download the site's text in large chunks.

Text this important shouldn't be an ordeal to read, and I am sure than the authors would never try to publish a journal article of comparable length in this format. They need to find a better way to present this material. For example, they could collapse the 100+ screens into just six pages--one for each of the six major sections--allowing users to view on screen, print, or save as needed.

Trivia Quiz

The MQ program ends with a quiz composed of a few dozen multiple-choice questions about the material covered in the main text. In several cases, I found none of the answers satisfactory. In other cases, the quiz presented two opinions as choices and demanded the "right" one. More frequently, the quiz posed silly questions, like asking about what provisions are included in which codes. Much as I believe in the power of historical knowledge, knowing whether the Nuremberg code included the right to withdraw from an experiment is of no obvious use to an ethnographer working today. And the quiz didn't even tell me which questions, if any, I got wrong. The quiz section was also heavy with typographical errors, making me wonder if it was written by someone other than the main authors of the MQ program. It is a shame to end such an innovative program on so sour a note.

There may be a bureaucratic need to include a quiz at the end; some administrator at MQ can now check that I have completed the training. But I see little pedagogic need for the quiz, since the program itself (after the medical ethics portion) is bound to be compelling to any ethnographer concerned with ethical research.

Ethical Education

Wynn, Mason, and Everett have put tremendous thought and effort into determining what ethnographers need to know about research ethics, and packaging that knowledge in a way that is respectful of researcher's learning, intelligence, curiosity, and desire to act ethically. They go too far when they claim that the ethics currently embraced by Australian and American anthropologists are universal to social scientists, and they at times make statements that lack nuance. But rather than proclaim these ideas as gospel, they have designed a training program that itself invites debate. I hope that other universities and other scholarly disciplines will follow their example.

Friday, April 10, 2009

Training Day

Peter Klein of the Organization and Markets blog offers a sad account of what it takes for a University of Missouri economist to gain permission to interview entrepreneurs or hand out surveys to corporate executives. Like many scholars across the country, he was directed to an online training system, which demanded that he provide correct answers to questions like the following:

32. The investigator is a 1/8th V.A. employee. She proposes to recruit MU outpatients into a study conducted exclusively at MU facilities. Which of the following groups must approve the research project before participants can be enrolled?

* The MU Health Sciences Center IRB
* The V.A. Research and Development Committee
* Both a. and b.
* Neither a. nor b.

While such knowledge may be of critical importance to health researchers at Missouri, it is irrelevant to social scientists not doing medical work. The lesson Klein takes away from such an experience is not that he must be sure to obey laws and ethics standards while doing his research, but that his campus IRB administrators do not respect him enough to provide relevant ethical training.

Administrators take note: you are making fools of yourselves, and earning your faculty's contempt.

See Comments Oppose New Regulations on Training.