United States of America Frees Oral History!

This morning sixteen federal agencies announced revisions to the Federal Policy for the Protection of Human Subjects, effective 19 January 2018. The final rule preserves and clarifies the NPRM’s deregulation of oral history. This is a great victory for freedom of speech and for historical research.

NSF Officer Misstates Belmont and Common Rule Standards

In the final contribution to the PS symposium, Lee Demetrius Walker, currently serving as program officer for the Political Science Program at the National Science Foundation, acknowledges the problems of applying a biomedical review system to social science. But he misstates the Belmont and Common Rule standards for assessing research.

[Lee Demetrius Walker, “National Science Foundation, Institutional Review Boards, and Political and Social Science,” PS: Political Science & Politics 49, no. 02 (April 2016): 309–12, doi:10.1017/S1049096516000263.]

Concern that IRBs will hinder field experiments

In his PS contribution, Brian Calfano worries that the Montana postcard fiasco will lead IRBs to impede political science field experiments, especially at teaching institutions.

[Brian R. Calfano, “‘I’ Does Not Mean Infallible: Pushing Back against Institutional Review Board Overreach,” PS: Political Science & Politics 49, no. 02 (April 2016): 304–8, doi:10.1017/S1049096516000251.]

Can community partners replace IRBs for field experiments?

In her contribution to the PS symposium, Melissa Michelson argues that “real-world practitioners” will often know more about relevant ethics and law than will the members of an IRB.

[Melissa R. Michelson, “The Risk of Over-Reliance on the Institutional Review Board: An Approved Project Is Not Always an Ethical Project,” PS: Political Science & Politics 49, no. 02 (April 2016): 299–303, doi:10.1017/S104909651600024X.]

Texas A&M IRB Imposed Review for Surveys of Public Officials

In their contribution to the PS symposium, Kenneth Meiera and Apolonia Calderona complain of IRB interference in work that is clearly exempt or not even human subjects research.

[Kenneth J. Meier and M. Apolonia Calderon, “Goal Displacement and the Protection of Human Subjects: The View from Public Administration,” PS: Political Science & Politics 49, no. 02 (April 2016): 294–98, doi:10.1017/S1049096516000238.]

IRB Chair: "Nobody Really Knows" If IRBs Do Any Good

A former chair of the University of Wisconsin-Madison Education and Social/Behavioral Science IRB guesses that IRB review “most likely” protects subjects from harm, but concedes that “nobody really knows.” He also notes that it consumes tens of thousands of hours of work, mostly by researchers, at his university each year.

[Kenneth R. Mayer, “Working through the Unworkable? The View from Inside an Institutional Review Board,” PS: Political Science & Politics 49, no. 02 (April 2016): 289–93, doi:10.1017/S1049096516000226.]

Yanow and Schwartz-Shea: IRBs Miss Their Targets

The first IRB related article in the April 2016 issue of PS, and the only one not formally part of the symposium on IRBs, is Dvora Yanow and Peregrine Schwartz-Shea, “Encountering Your IRB 2.0: What Political Scientists Need to Know.” This essay is intended as an introduction to IRB issues for political scientists, and therefore presents material that will be familiar to readers of this blog.

In addition to this helpful introduction, Yanow and Schwartz-Shea make an interesting point about the scope of IRB review: on the one hand, it can be under inclusive, failing to cover serious ethical questions. On the other hand, mission creep continues apace, as universities impose restrictions not dictated by ethics or law.

[Dvora Yanow and Peregrine Schwartz-Shea, “Encountering Your IRB 2.0: What Political Scientists Need to Know,” PS: Political Science & Politics 49, no. 02 (April 2016): 277–86, doi:10.1017/S1049096516000202.]

Political Scientists Consider "Local Control and Realities" and IRB Oversight

The April issue of PS: Political Science & Politics features several items about political science and IRBs. Here’s a list; commentary will follow if I find the time.

• Dvora Yanow and Peregrine Schwartz-Shea, Encountering Your IRB 2.0: What Political Scientists Need to Know

• Valerie Martinez-Ebers, Introduction.

• Kenneth R. Mayer, Working through the Unworkable? The View from Inside an Institutional Review Board.

• Kenneth J. Meier and M. Apolonia Calderon, Goal Displacement and the Protection of Human Subjects: The View from Public Administration.

• Melissa R. Michelson, The Risk of Over-Reliance on the Institutional Review Board: An Approved Project Is Not Always an Ethical Project.

• Brian R. Calfano, “I” Does Not Mean Infallible: Pushing Back against Institutional Review Board Overreach.

• Lee Demetrius Walker, National Science Foundation, Institutional Review Boards, and Political and Social Science.

Political Scientists Complain about Lost Exemption

Political scientists and law professors are protesting the proposed elimination of the current exemption for research about public officials and candidates for office. The drafters may have eliminated the exemption by mistake, thinking that such research was exempted by other provisions of the proposed rule, which turns out not to be the case.

Montana Political Scientist: IRB Process Is "Cumbersome, Inadequate" for Political Research

The Montana Commissioner of Political Practices (COPP), Jonathan Motl, has determined “that there are sufficient facts to show that Stanford, Dartmouth and/or its researchers violated Montana campaign practice laws requiring registration, reporting and disclosure of independent expenditures” in October 2014, when they mailed pre-election postcards to 102,780 registered voters. The postcards were part of an effort to see if certain claims about candidates’ place on an ideological spectrum would affect voting patterns.

As part of the investigation, Motl engaged Carroll College political science professor Jeremy Johnson to determine whether the Stanford and Dartmouth researchers had violated IRB requirements. Johnson finds that they did, but also that the Dartmouth IRBs could have approved the studies without addressing the most serious ethical issues they raised. “The fundamental problem with the IRB process,” he writes, “is the narrow focus on protecting the individual subject. Concerns about human subjects in the aggregate often do not even occur to researchers, faculty, and staff involved in the IRB.”

[McCulloch v. Stanford and Dartmouth, Commissioner of Political Practices of the State of Montana, No. COPP 2014-CFP–046, Decision Findind Sufficient Facts to Demonstrate a Violation of Montana’s Campaign Practice Laws, 11 May 2015. h/t Chris Lawrence.]

"The Freedoms We Are Committed to Protect"

The Hinckley Institute of Politics, University of Utah, has posted a videorecording of my lecture there last month.

"'The Freedoms We Are Committed to Protect': Political Science, Academic Freedom, and Institutional Review Boards in Historical Perspective.” Keynote address, Symposium on Field Research and US Institutional Review Board Policy, University of Utah, March 2014. http://youtu.be/uwIFyDgLklU

Symposium: Field Research and US Institutional Review Board Policy

This month the Political Science Department, University of Utah, will host a symposium entitled, "Field Research and US Institutional Review Board Policy." Sponsored by the Betty Glad Memorial Fund, the symposium will take place March 20 [8:45 am - 5:30 pm] and March 21 [9:00 am - 3:30 pm].

The description follows:

Belfast Project: No lawyers, few historians (and no IRB)

Discussions of the ill-fated Belfast Project at Boston College often frame the issue as what can happen to an oral history project in the absence of IRB oversight. But a recent account of the project in the Chronicle of Higher Education, as well as subsequent discussion, suggests that the real problem was a lack of involvement by lawyers and historians.

[McMurtrie, Beth. “Secrets From Belfast.” The Chronicle of Higher Education, January 26, 2014.]

University of Utah Plans Symposium on IRB Policy

On March 20 and 21, the University of Utah Political Science Department will host a symposium, Field Research and US Institutional Review Board Policy.

I will give the keynote address, "'The Freedoms We are Committed to Protect': Political Science, Academic Freedom, and Institutional Review Boards in Historical Perspective."

CU-Boulder Tells Faculty to Consult IRB Before Teaching

A spokesman for the University of Colorado at Boulder has recommended that university faculty consult the IRB before teaching.

The comment concerns sociology professor Patti Adler's announcement that she plans to retire early rather than risk being fired for classroom teaching that might make students uncomfortable.

[Scott Jaschik, “Tenured Professor at Boulder Says She Is Being Forced out over Lecture on Prostitution," Inside Higher Ed, December 16, 2013. http://www.insidehighered.com/news/2013/12/16/tenured-professor-boulder-says-she-being-forced-out-over-lecture-prostitution.]

Australian Political Scientist: "Causing Harm . . . May Be the Whole Point"

Anthony Langlois, Associate Professor in the Department of International Relations at Flinders University, Adelaide, Australia, and a former member and chair of that university's Social and Behavioural Research Ethics Committee, finds that Australia's 2007 National Statement on Ethical Conduct in Human Research is an improvement over earlier policies, but that it still leaves an "ill fit between the requirements of [the medical] model of research ethics review and the nature of humanities and social sciences research."

[Anthony J. Langlois, "Political  Research  and  Human  Research  Ethics  Committees," Australian Journal of Political Science 46 (2011): 139-154, DOI: 10.1080/10361146.2010.544287. Also available as a preliminary preprint. Thanks to Professor Langlois for mentioning the essay on socialsciencepsace.com.]

La Noue Reviews Ethical Imperialism

Political scientist George La Noue terms Ethical Imperialism "a powerful indictment of the IRB regime."

[George R. La Noue, Review of Zachary M. Schrag, Ethical Imperialism: Institutional Review Boards and the Social Sciences, 1965-2009, Law & Politics Book Review 20 No. 11 (October, 2010): 616-618.]

La Noue, who has himself written about the IRB controversy, notes that "Universities might seem to be a most unlikely place to welcome and implement a process that is in effect a form of prior censorship. Reconciling the IRB process with legal or professorial concepts of academic freedom is extremely difficult." He finds that "Schrag provides a carefully researched and well written historical perspective providing all members of the academy with essential information to reconsider the role of IRBs."

La Noue calls for more study of the constitutionality of current IRB regulations and practices, a subject I would prefer to leave to the law professors. He also concludes that

What is missing is comparative empirical research about the standards and procedures of a variety of IRBs in different settings. While it seems intuitively unlikely that the process is always fair, objective and consistent, from IRB committee to committee, campus to campus, beyond anecdotes what proof exists? Without an appropriate factual basis, courts would struggle with both the compelling interest and narrow tailoring prongs that constitute the strict scrutiny test that should apply to censorship. Schrag’s book provides a necessary and very carefully researched historical context for the debate about IRBs, but the next step needs to be taken by professional associations and social scientists to study their actual practice to see if the current system can be improved.

I concur, though I would suggest that we in fact need two separate branches of such emprical study. One would continue the work of Maureen Fitzgerald and Laura Stark, both of whom have observed committees in action without finding huge variation from campus to campus, or even--in Fitzgerald's case--country to country.

A second branch would look at the development of human research protections policies. Read through this blog, and you will find enormous variation in casts of characters involved in shaping university policies on human subjects, from research offices that feel free to make up whatever rules they want, to the participation of university-level faculty committees, to the involvement of departments most affected by a given policy.. I don't know of any scholarship that has examined this variation in depth.

I hope that other scholars heed Professor La Noue's call.

Political Scientists to the Rescue?

The final essay in the PS symposium is Sue Tolleson-Rinehart, "A Collision of Noble Goals: Protecting Human Subjects, Improving Health Care, and a Research Agenda for Political Science."

Tolleson-Rinehart addresses the question of quality improvement in health care, such as the checklist publicized by Atul Gawande. As she notes, her "essay is not about the influence of IRBs on political science research" and is therefore largely outside the scope of this blog. That said, she makes some observations relevant to the regulation of social science research.

While sympathetic to individual IRB members, Tolleson-Rinehart takes a dim view of the system as it now operates:

IRBs are understandably, and necessarily, driven by their limiting case: the possibility of invasive or dangerous procedures performed with vulnerable individuals or populations, without adequate regard for the hallmarks of protection, respect for persons, beneficence and nonmaleficence, and justice elucidated in what is widely known as the Belmont Report, and adopted from the time of the Belmont Report’s release as our fundamental ethical principles. It is not surprising, given IRBs’ role as the protector of the vulnerable, that the general IRB perspective on “minimal risk” and risk-benefit comparisons is a conservative one.

IRBs are also terrified. All IRB professionals I know work in real fear that their IRBs could be the next ones to cause an entire university’s research to be shut down. Shutdowns in recent years at Harvard, Duke, and Johns Hopkins give IRBs every reason to be fearful of making what OHRP considers to be an error. The reasonable suspicion that researchers regard IRBs as obstacles to, rather than facilitators of, research, must further IRB professionals’ sense of being embattled.

Reviewers on IRB committees are our very hardworking colleagues who are usually not given adequate release time to meet their committee responsibilities, and who are not able to benefit from truly extensive and nationally standardized training, nor do they have anything like a consensus metric for evaluating the spectrum of risk in different research contexts and for different populations.

All these sources of strain might determine the conservative approach to human subject protections. When social science research (including quality-improvement research) occurs in a biomedical context, or when health care and health policy require evaluation, the conservative stance can become dysfunctional. IRB assessments of my own students’ work provide a clear example of one of the ironic and unintended consequences of the absence of agreed upon and broadly understood metrics for assigning risk in different research contexts. IRBs have a comparative lack of familiarity with how social science methods— such as those used in quality-improvement research—may differ from some other methods of clinical research in the risks they pose to subjects. (508)

She adds that while her own students submit "substantially similar" protocols, their

IRB determinations range from declarations that the project is “not human subjects research” at all to “research that requires full consent,” with every intermediate determination. I frequently have students working simultaneously on very similar projects, one of whom must go through a tedious consenting process taking as much as four to five minutes of the beginning of telephone interviews (with the busy elites at the other end chafing at these long and unnecessary prefaces to the first question), while another student researcher is not required to secure any kind of consent at all. The single source of variation across these cases is not the research question or the method, but the IRB reviewers and their familiarity ~or lack thereof ! with in-depth interviewing and the unique protections already available, via one’s position, to the “powerful research subject.” (509)

This is damning enough, but Tolleson-Rinehart insists that the "point of this vignette is not to criticize IRBs." (509) Rather, she argues that

political science is well prepared to analyze and make normative (but evidence-based) recommendations about the politics of human subjects research. We can help define what it is, and the circumstances under which it is generalizable knowledge, even though it may not necessitate a conservative approach to protections. We can construct frameworks to achieve a more precise understanding of how to balance risks and benefits. Those frameworks might even lead us to formulate what would amount to a multidimensional risk scale. Finally, political science can contribute to the construction of theoretical and methodological underpinnings for the content of truly national standards for IRB training curricula. These would improve IRB reviewers’ understanding of different research methods to go beyond mere compliance with federal regulations and become real resources and decision aids for hard-pressed reviewers who may have to evaluate research they aren’t familiar with. (509)

Finally, Tolleson-Rinehart notes that while the Association for the Accreditation of Human Research Protection Programs and Public Responsibility in Medicine and Research mean well, both emphasize regulatory compliance over actual research ethics. She argues that political scientists can go beyond compliance questions to work on "a common epistemology of the philosophical, ethical, and political foundations of human subjects research." (510)

All of this sounds fine, and I hope that Tolleson-Rinehart and her colleagues get to work on her agenda. But as my recent exchange with Levine and Skedsvold suggests, the most immediate question for political scientists may be to figure out how to make the regulatory system more responsive to developments in research. We seem stuck with a 1974 law and 1991 regulations that cannot be changed, even when everyone agrees they need updating.

Can We Patch This Flat Tire?

The fourth article in the PS symposium is Felice J. Levine and Paula R. Skedsvold, “Where the Rubber Meets the Road: Aligning IRBs and Research Practice.” Both authors been involved in IRB debates for several years, and this article reflects their sophisticated understanding of some of the issues involved. But for an article published in a political science journal, it is disappointingly insensitive to the power dynamics that govern IRB-researcher relationships.

Unlike symposium participants Tony Porter, Dvora Yanow and Peregrine Schwartz-Shea, Levine and Skedsvold do not question the premise that IRBs help promote ethical research. Instead, they assert that there is no fundamental conflict between IRBs and social science researchers: "federal regulations, professional ethics codes, and research practice may have shared goals but tend to speak with different languages—creating frustration and skepticism in a system that could potentially work quite well if transformations are made." (502) Based on that assertion, they suggest four such transformations, ranging from the bold to the timid.

The Biomedical Ethics Juggernaut

The third contribution to the PS symposium is Tony Porter, "Research Ethics Governance and Political Science in Canada."

Porter laments that "the history of research ethics governance in Canada reveals recurrent concerns expressed by political scientists and other SSH [social sciences and humanities] researchers that indicate the inappropriateness of the [ethics] regime for SSH research, and that also create the impression that the regime is a juggernaut that continues on its trajectory, relatively impervious to criticism." (495)

Porter then offers a helpful capsule history of the debates leading up to Canada's present policy statements. From an American perspective, they look pretty good. In contrast to the Belmont Report, which calls for informed consent and harms-benefit assessment without specifying the types of research to which it applies, Canada's Tri-Council Policy Statement declares:

certain types of research— particularly biographies, artistic criticism or public policy research—may legitimately have a negative effect on organizations or on public figures in, for example, politics, the arts or business. Such research does not require the consent of the subject, and the research should not be blocked merely on the grounds of harms-benefits analysis because of the potentially negative nature of the findings. (496)

Unfortunately, Porter finds that in practice, research ethics boards ignore such guidance. For his own article, he was asked to specify questions in advance, destroy data, and write long explanations of his research plans. And he warns of even stricter regulation ahead.

Porter attributes the imposition of biomedical ethics and regulation on non-biomedical research to the clout that biomedical researchers have in government and universities. There are more of them, they have more money, and they care more about ethics--since they face more serious ethical challenges. As a result, "the growth of a biomedically oriented but unified research ethics regime has appeared as a seemingly unstoppable trend in Canada." (498) Rather dismally, Porter suggests that the only thing that will stop that trend is its own ability to alienate researchers until "opposition on the part of SSH researchers will increase and the legitimacy of the arrangements will be damaged, as will the ability of the regime to elicit the degree of voluntarism and acceptance that is needed to sustain it." (498)

Perhaps for lack of space, Porter does not consider another possibility: that the social sciences will internalize the medical ethics implicit in the "unified research ethics regime." The American Anthropological Association took a big step in this direction in 1998, with the adoption of a code of ethics that comes close to rejecting the idea that research "may legitimately have a negative effect on organizations or on public figures." If the ethics regime grows stronger in Canada and elsewhere, and more social scientists follow the AAA's line, it may be that young people interested in "critical research," as Porter puts it (496), will seek careers in journalism, rather than in university scholarship. To use a Canadian example, if Russel Ogden were writing for a newspaper, no one would be blocking his research.