When Seligson is Non-Seligson

The first article in the July 2008 PS symposium is Mitchell A. Seligson's “Human Subjects Protection and Large-N Research: When Exempt is Non-Exempt and Research is Non-Research." While it's great to have someone interested in the contradictions of IRB regulations, the absurdity of the present regime seems to have left Seligson hopelessly confused, and his incoherent essay calls for both expansion and contraction of IRB authority.

Rather than trying to outline his argument, let me just list some of the questions to which he poses contradictory answers.

1. Should social science and humanities research follow the Belmont Report?

Early in his essay, Seligson attacks the Belmont Report as irrelevant to social science research, especially survey research. He particularly dislikes its call for an assessment of risks and benefits, noting

the problem of assessing risk is especially vexing for all of those who rely on large-N studies, typically in the field of survey research. Ironically, when only a handful of subjects are used in a campus laboratory-based experiment, the IRB is likely to approve the project with no objection. But survey research, which invariably relies on large-N studies, is viewed with suspicion by many IRBs simply because the risk, however small, is seen as being replicated 1,000 or more times, since most samples strive for confidence intervals of 63% or better. Protocol analysts, who are used to seeing laboratory experiments and focus groups with samples of fewer than 100, are often taken aback when they confront the large sample sizes inherent in most survey research. And when they do, they question why such a large sample is needed. As a result, it is not at all uncommon to have IRB protocol analysts ask survey researchers to cut down their sample sizes. (479)

He also is skeptical of the Common Rule, especially its protections for pregnant women--irrelevant and damaging to survey research. And he quotes--seemingly with approval--the AAUP's 2006 recommendation "that research whose methodology consists entirely of collecting data by surveys, conducting interviews, or observing behavior in public places be exempt from the requirement of IRB review.”

But then Seligson turns around, lamenting that "historians are not only exempt from IRB control, they have no requirement or even need to take human subjects protection training and pass tests on their knowledge of the principles and rules. Literature faculties often have no knowledge at all of human subjects protection." (480) He wants "faculty members in a broad range of institutions to familiarize themselves with the IRB regulations and to take the tests to demonstrate their knowledge of same," including "the Belmont principles." (482)

Why? Why should faculty members be required to familiarize themselves with guidelines that Seligson has told us are inapplicable to their work? Does he just want company in his misery?

2. Can researchers be trusted?

Seligson thinks that IRB regulations did not help survey research, because

Long before human subjects regulations and the invention of IRBs, survey researchers in all fields instinctually knew that by guaranteeing anonymity they would encourage frankness on the part of respondents. . . . Political scientists who carry out surveys have been aware for decades of the importance of guaranteeing anonymity to their subjects. (480)

If this track record weren't enough, he notes that governments and universities trust political scientists to behave ethically in other aspects of their work.

Even though political scientists conducting educational tests and surveys are exempt from federal regulation, they are not, after all, exempt because the federal government believes we cannot be trusted. What is so strange here is that in countless other important ways, we are trusted by that same federal government. When we grade tests taken by our students, we are not allowed to discriminate on the basis of race, creed, national origin, sexual preference, etc. Yet we are not asked to sign a statement saying that we will not discriminate before ~or indeed after! we grade each exam or before we determine final grades. We hold office hours, but are not asked to submit an application prior to each office hour, not even prior to the start of each term, to the affirmative action offices on our campuses that we will not sexually harass students. We submit articles to conferences but are not asked to submit signed statements saying that we did not plagiarize the material. (481)

Since political scientists have proven more or less trustworthy in these areas, Seligson wants IRBs "to stop assuming, . . that we are all guilty of violations of human subjects rights unless we can prove otherwise." (482)

That's all very nice, but he's unwilling to extend the trust to researchers in other fields. He writes,

some humanists may be naive about the risks involved in disclosing names of subjects. One can imagine many kinds of risk to respondents. One such risk is dismissal of employment from an employer who either might not like the views expressed in the oral history or testimonio or deems them harmful to the company’s welfare. Potential employers might look at the oral history information and deny a position based on the statements contained therein. Another risk could be ostracism at work or in one’s neighborhood for expressing politically unpopular views. One can even imagine law enforcement officials using oral histories to prosecute individuals for revelations that suggest criminal behavior. (480)

In other words, Seligson does not trust interview researchers to have the same instinctual knowledge of ethics he ascribes to survey researchers, he ignores oral historians' sixty-year record in favor of hypothetical abuses, and he assumes historians are guilty of violations of human subjects rights unless we can prove otherwise. Perhaps he wants us to get approval before grading tests as well.

3. Can IRBs be trusted?

Overall, Seligson takes a dim view of those in charge of human subjects regulations, whom he terms "overzealous bureaucrats, both federal and on campuses," and wants retrained. (482) He even relays the follwoing anecdote:

A very senior IRB official at one university, in order to impress upon a political science faculty member his omnipotence, asked, “Do you ever use the library to read books about President Bush?” When the response was affirmative, he said, “Unless you file for IRB approval before opening those books, you will be held in violation, since Bush is a human, is living, and the books almost certainly contain personal information.” (480)

I'm willing to believe a lot of bad things about IRBs, but even I can't swallow a story like this without names and dates attached.

Yet while portraying IRB officials as power-mad bureaucrats, Seligson wants to expand their jurisdiction "to cover all studies of any kind that obtain data on living humans." (482) Wouldn't that include a book about President Bush?

Seligson concludes that "the roadmap to the future should be clear." Maybe it should be, but this article isn't helping. Fortunately, the other essays in the symposium are better researched and reasoned.