A Horror Anthology

Mark Kleiman takes on IRBs at The Reality-Based Community. On April 14 he asked his readers for IRB horror stories, and on May 2 he posted some of the responses.

The saddest concerns a group of law students who wished "to send testers of different races in different styles of clothing to the restaurant over some period of time to test whether they enforced their dress code in a discriminatory manner." Law school administrators told them they would have to secure IRB approval. This discouraged the students, who did not want to go through the time and effort of the approval process.

This was not the intent of the National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research. At its 15 April 1978 meeting, the commission discussed just such a scenario (pp. II-5 to II-21 of the transcript), and all the members seemed to agree that such testing for discrimination should not require IRB review. But, as I've noted before, the commission wrote a definition of human subjects research that plausibly includes a great deal of activity the commission did not seek to regulate. Thirty years later, justice suffers as a result of the commission's sloppiness.

NOTE: In honor of Professor Kleiman's search, I have gone back through this blog to add the "horror stories" tag to some posts that should have had it to begin with. Clicking on that tag now yields more than a dozen posts, with even more documented horror stories.

Journal of Policy History

The Journal of Policy History has published my article, "How Talking Became Human Subjects Research: The Federal Regulation of the Social Sciences, 1965–1991." As permitted by the transfer of copyright, I have posted a PDF on my personal website: http://www.schrag.info/research/howtalking.html.

Not much has changed since I posted a version on SSRN in April 2008. The major changes come in the "medical origins" section; the earlier draft underestimated the strength of social scientists' opposition to IRB rules in the late 1960s. Also, the new version better explains the origins of Ithiel de Sola Pool's concern about IRBs (see p. 18).

DeGette Still Doesn't Get It

Representative Diana DeGette (D-CO) has introduced the Protection for Participants in Research Act (H.R. 1715), which would impose IRB requirements on all human subject research supported by the federal government or affecting interstate commerce.

As amp&rsand notes, this is the sixth time DeGette has introduced this bill. And I don't think that counts earlier submissions of similar bills by Senator John Glenn. None of these previous efforts went far, so there's no particular reason to fear this bill's passage.

Still, it is disappointing that DeGette has introduced this bill six times without understanding its potential consequences. Her press release states that “I think one thing we can all agree on in a bipartisan way is that we need to encourage medical experimentation but we need to do it in a way that both protects the patients and gives them informed consent about what they are getting into," as if the bill would affect only medical experimentation. It points to medical trials in 1999 and 2006 as evidence of insufficient oversight, and argues that "research is the key to innovation and discovery, including curing deadly disease." Nowhere in the press release is a hint that DeGette understands that her bill would outlaw most journalism, not to mention further inhibiting social science and humanities research.

Thirty-five years after the passage of the National Research Act, Congress still doesn't know what it has done.

Bonus question for Rebecca Tushnet's 43(B)log: Is the copyright claim on DeGette's press release--the work of a federal employee in her official capacity--illegal, or merely false?

Update 8 May 2009: A correspondent notes that the copyright statement is no longer on the site. If DeGette removed the statement in response to this blog, good for her. I have a PDF of the press release as it appeared on May 4, if anyone is interested.

UMKC's Respectful Oral History Policy

The University of Missouri-Kansas City (UMKC) has posted a promising new policy: "Social Sciences IRB and Oral History."

The policy has a number of elements that set it apart from the typical university policy, which seeks to cram oral history into a system designed for medical experimentation. Instead, it adapts only those elements of the medical IRB system that encourage historians to follow their own discipline's ethics and best practices.

I suggest that readers of this blog read the whole policy, but here are some highlights:

1. Respect for Critical Inquiry

As I have written repeatedly on this blog, historians do not take the Hippocratic Oath, and should not promise not to harm the people they interview. Any IRB that imposes the Belmont Report on historians is asking them to forswear their own ethics.

UMKC understands this. Its policy notes that

akin to a journalist or lawyer, an historian is also responsible to a wider public to recover a shared past “as it really happened.” In keeping with the public role of an historian in a democratic society, these responsibilities, especially when conducting narrative interviews, can necessitate a confrontational style of critical inquiry. So while historians do not set out to hurt their interviewees, oral historians are expected to ask tough questions in their interrogation of the past.

2. Respect for Peer Review

The UMKC neither subjects oral historians to the whims of board members unfamiliar with their field, nor does it leave them on their own. Instead, it offers scholars a number of relevant readings, including publications of the Oral History Association, and then encourages them to talk to colleagues knowledgeable about interviewing:

After reviewing these resources on their own, the researcher is strongly encouraged to discuss their research protocol with peers before implementing their research protocol. In some cases, peer review by members of one’s own department would be most useful; in other cases, a researcher might be better served by seeking review from a colleague in a different department.

To foster these kinds of conversations among the faculty, the Social Sciences IRB Subcommittee for Oral History will hold two meetings per semester . . . to discuss “Best Practices” in oral history. Faculty experts in oral history will guide these conversations . . . These meetings are designed to meet the needs of researchers seeking advice and peer review for their research protocols. They are also designed to meet the needs of Chairs and/or designees interested in learning how to advise researchers in their departments to make responsible decisions regarding oral history.

3. Respect for OHRP's Pledge

UMKC takes seriously the carefully negotiated 2003 agreement between the American Historical Association and the Oral History Association and OHRP, even posting a copy on its website. The university elaborates on that agreement:

At UMKC, we draw a distinction between idiographic research that uses oral histories to describe the unique story of some particular social group or individual, which does not constitute “human subjects research”; and nomothetic research that employs oral histories in the hopes of contributing to a general theoretical or comparative debate about the human nature or behavior, which does fall under the category of “human subjects research."

While I confess that the terms idiographic and nomothetic are not in my working vocabulary, I believe they do express a real difference between the ethics of oral historians and those of other scholars. If one is interested in a general theoretical or comparative debate about the human nature or behavior--as many social scientists seem to be--then it makes less sense to single out individuals for potential honor or calumny. Writing about unique individuals or groups changes one's responsibility toward the individuals interviewed.

4. Respect for Researchers

Policies like UCLA's infantilize researchers, making them submit every judgment to an administrator. By contrast, UMKC trusts its scholars:

The bottom line is that the researcher makes these determinations in careful consultation with the Chair of the department or another official designee appropriate to the kind of study being planned. Together this determination is based on shared understanding of all relevant guidelines and their shared expertise in their specialized field of scholarship.

5. Respect for the IRB

Even as it empowers historians, the UMKC policy keeps the IRB involved, making it a resource, rather than an obstacle. Researchers still have to learn something about human subjects regulations, and they must complete a form explaining why they have determined that their policy does not fall under federal regulations.

(The form's demand for an explanation of "no more than 1500 characters" sounds suspiciously bureaucratic, but it's a good length for the presentation of a single idea--about the same as the 150-word limit for a New York Times letter to the editor.)

More importantly, the frequent meetings of the Social Sciences IRB Subcommittee on Oral History suggest that some scholars at UMKC have devoted their time to helping colleagues deal with the real ethical challenges of oral history.

The website explaining the policy notes that it was developed by "a group of faculty and administrators involved with the Social Science Institutional Review Board (SSIRB) . . . with input from members of the SSIRB, the College of Arts & Sciences, and the Faculty Senate at UMKC." I congratulate all the scholars and administrators who developed this innovative system, and I hope it works as well in practice as it reads on the screen.

With this policy, UMKC joins Amherst College, Columbia University, the University of Michigan-Ann Arbor, and the University of Nebraska-Lincoln Policy on a small but growing list of schools that have adopted OHRP's 2003 position removing most oral history research from IRB jurisdiction. Five schools not very many, but it's five more than the AHA could find in February 2006. Who will be number six?

Deregulation "Is Not Going to Happen"

Linda Shopes kindly alerts me to the April 20 issue of COSSA Washington Update, the newsletter of the Consortium of Social Science Organizations, which reports on an April 1 meeting of the National Academies’ Board on Behavioral, Cognitive, and Sensory Sciences, at which IRBs were discussed.

Here's the key passage:

Philip Rubin, CEO of Haskins Laboratories in New Haven, CT, and former director of the National Science Foundation’s (NSF) Division of Behavioral and Cognitive Sciences, chairs the Board. He began the session with a review highlighting the difficulties social/behavioral researchers have had with the current system under the Common Rule regulation and its interpretation by campus Institutional Review Boards (IRBs). Complaints have been loud, but mostly anecdotal . . . Once again the bottom line is that despite efforts by Joan Sieber and the Journal of Empirical Research on Human Ethics, which she edits, there are still large gaps in our empirical knowledge of how the system works for social and behavioral scientists.

Rubin was followed by Jerry Menikoff, new head of the U.S. government’s Office of Human Research Protections (OHRP). Menikoff announced that he was all for “flexibility” in the system and that “changes can be made.” He also endorsed conducting more research. He rejected the arguments of the American Association of University Professors and Philip Hamburger of Northwestern University Law School that IRBs violate researchers’ first amendment rights. He acknowledged the importance of expedited review, but stated quite clearly that “removing minimal risk research from the system is not going to happen.”

I don't want to make too much of these comments; an OHRP spokesperson tells me that they were an extemporaneous response to Rubin, and not prepared remarks. Still, I am disappointed. Menikoff's comments suggest a retreat from his earlier concession that "flexibility" often can be code for arbitrary power. And it's a pity for a public official to insist that a given policy "is not going to happen" even as he endorses more research. Wise governance depends on making policies after finding facts, not before.

Macquarie's Innovative Ethics Training

In previous posts and my 2007 essay, "Ethical Training for Oral Historians," I have complained about standardized, medicine-centric ethics training systems like the CITI Program and called for training programs better tailored to individual disciplines.

Lisa Wynn of Macquarie University (also known as MQ) has alerted me to just such a program she created with Paul H. Mason and Kristina Everett. The online module, Human Research Ethics for the Social Sciences and Humanities, has some elements that I find inappropriate. Overall, however, it is vastly superior to the CITI Program and comparable ethics programs I have seen, and it deserves attention and emulation.

Training Day

Peter Klein of the Organization and Markets blog offers a sad account of what it takes for a University of Missouri economist to gain permission to interview entrepreneurs or hand out surveys to corporate executives. Like many scholars across the country, he was directed to an online training system, which demanded that he provide correct answers to questions like the following:

32. The investigator is a 1/8th V.A. employee. She proposes to recruit MU outpatients into a study conducted exclusively at MU facilities. Which of the following groups must approve the research project before participants can be enrolled?

* The MU Health Sciences Center IRB
* The V.A. Research and Development Committee
* Both a. and b.
* Neither a. nor b.

While such knowledge may be of critical importance to health researchers at Missouri, it is irrelevant to social scientists not doing medical work. The lesson Klein takes away from such an experience is not that he must be sure to obey laws and ethics standards while doing his research, but that his campus IRB administrators do not respect him enough to provide relevant ethical training.

Administrators take note: you are making fools of yourselves, and earning your faculty's contempt.

See Comments Oppose New Regulations on Training.

Deadline Extended for TCPS Comments

John Lowman kindly alerts me that Canada's Interagency Advisory Panel on Research Ethics has extended the deadline for comments on the draft second edition of the Tri-Council Policy Statement: Ethical Conduct for Research Involving Humans (TCPS). Comments will now be accepted through 30 June 2009, though the PRE encourages comments by March 31, since the next round of revision will begin in April.

An official announcement of the deadline extension is online at the PRE's French-language website. I could not find an English-language version on the PRE website, but the University of Western Ontario has posted one.

A form for online comments, and instructions for submitting comments by mail, fax, or e-mail, is online.

I have sent in a version of the comments posted on this blog. As I prefaced my comments to the PRE, I write as a non-Canadian. But the regulation of research ethics is an international endeavor. Just as TCPS draws heavily from the Belmont Report and 45 CFR 46, so can we expect TCPS to influence American policy and guidance. I therefore consider myself to have some stake in the outcome of the TCPS revision.

Canadian Criminologists Decry TCPS Draft

Back in January, I mentioned the release of the Draft 2nd Edition of the Tri‐Council Policy Statement: Ethical Conduct for Research Involving Humans, prepared by Canada's Interagency Advisory Panel on Research Ethics, or PRE.

Ted Palys and John Lowman of the School of Criminology, Simon Fraser University, kindly alerted me to their critique of the draft, or TCPS-2, as they term it. (They even more kindly cited this blog in their work.) They find that TCPS-2 "poses a significant threat to academic freedom in Canada." (3)

Their 20-page critique, "One Step Forward, Two Steps Back: Draft TCPS-2’s Assault on Academic Freedom," is all meat and no fat, and I recommend that it be read in its entirety. But here are a few salient points.

Less Flexibility, More Freedom

Defenders of the present IRB system often boast of the "flexibility" offered by current regulations. (See, for example, Dr. Jeffrey Cohen's report from the November PRIM&R meeting.)

But flexibility--when combined with the possibility of punishment--can actually empower censorship. Here is how Human Rights Watch describes an analogous system, China's censorship of the Internet:

The display of politically objectionable content can result in reprimands to company management and employees from the MII, the State Council Information Office, the Communist Party's Propaganda Department, and/or various state security organs, accompanied by warnings that insufficient controls will result in revocation of the company's license. In order to minimize reprimands and keep their licenses in good standing, BBS and blog hosting services maintain lists of words and phrases that either cannot be posted or which cause monitoring software to "flag" the content for manual removal by employees.

Search engines likewise maintain lists of thousands of words, phrases and web addresses to be filtered out of search results so that links to politically objectionable websites do not even appear on the search engine's results pages, even when those websites may be blocked at the backbone or ISP level . . . Such lists are not given directly to Internet companies by the Chinese government; rather, the government leaves the exact specifics and methods of censorship up to companies themselves. Companies generate their "block-lists" based on educated guesswork plus trial-and-error: what they know to be politically sensitive, what they are told in meetings with Chinese officials, and complaints they may receive from Chinese authorities in response to the appearance of politically objectionable search results.

But the complicity of companies is even more direct: they actually run diagnostic tests to see which words, phrases, and web addresses are blocked by the Chinese authorities at the router level, and then add them to their lists, without waiting to be asked by the authorities to add them. And because they seek to stay out of trouble and avoid complaints from the authorities, many businesspeople who run [Internet Content Providers] in China confess that they are inclined to err on the side of caution and over-block content which does not clearly violate any specific law or regulation, but which their instincts tell them will displease the authorities who control their license. In all these ways, companies are doing the government's work for it and stifling access to information. Instead of being censored, they have taken on the role of censor.

In other words, by keeping secret the exact terms that will trigger a license revokation, the Chinese government achieves more censorship than it could by publishing a list of forbidden terms, and it makes Google, Yahoo!, and other U.S. companies complicit in the censorship. Similarly, OHRP's vagueness about what will trigger a shutdown fails to assure universities that they can safely deregulate research, so universities restrict research that should be exempt from review.

Fortunately, OHRP's new director, Jerry Menikoff, understands this. In the Winter 2009 issue of AAHRPP Advance he writes,

We often hear that it’s better not to provide specific guidance—that the absence of guidance allows people greater flexibility in interpreting the regulations. In my experience, the opposite can be true. Guidance can empower individuals and advance both research and research protections. In the absence of guidance, people tend to be reluctant to take certain actions out of fear that they are violating the rules. In some instances, important research is not even attempted, all because of a misunderstanding. Guidance could eliminate the misconception and clear the way for research.

I'm delighted that Dr. Menikoff takes this approach, and I look forward to more specific guidance from OHRP that would clear the way for research.

[Thanks to Rob Townsend for altering me to Menikoff's comments.]