PRIM&R and SACHRP Attack Social Science Exclusion

In their comments on the NPRM, SACHRP and PRIM&R oppose the proposed exclusion of “Research, not including interventions, that involves the use of educational tests (cognitive, diagnostic, aptitude, achievement), survey procedures, interview procedures, or observation of public behavior (including visual or auditory recording) uninfluenced by the investigators” (§ ll.101(b)(2)(i)); they want such research to be moved from the excluded to the exempt category. But they differ in what they think the consequences of such a move would be; SACHRP thinks that researchers would face low barriers, while PRIM&R sees a chance for its members to continue to exert more control than is authorized by regulations. Both groups fail to represent the researchers most likely to conduct these kinds of studies.

I Agree With PRIM&R on Something

PRIM&R has posted its draft comments on the NPRM. I have found something to agree with.

PRIM&R writes:

Over the past 25 years, the difficulty of getting all of the agencies to agree on the Common Rule was cited as the grounds for concluding that changing any provisions would require a Herculean effort. It may well be that revising the entire Common Rule at once is a task that can only be undertaken once in a generation.

But if the current NPRM, issued four years after the ANPRM to revise the Common Rule (July 26, 2011), underlines how daunting a comprehensive revision can be, it also shows that the best, evidence-based revisions will not emerge from a process that tries to make all needed revisions at once. By perpetuating the view that regulations cannot be revised when needed, the current approach risks locking in place for another 25 years a number of provisions in the NPRM that even its most enthusiastic supporters admit are flawed in important ways.

Instead, we recommend that the relevant agencies pursue an issue-by-issue approach in which it is possible to “drill down” on an issue, to consider all the sections of the regulations where it arises and all the evidence that is available—and that is needed—to resolve it well. Further, the agencies should address each issue more openly, developing and relying upon evidence, and allowing consensus to emerge and revisions to be crafted that will work well for all stakeholders. This approach would not only produce better results, but would signal that, once adopted, any provision can be changed if it does not work as well as intended.

Yes, indeed.

I don’t go so far as to agree with PRIM&R that the proposed rule should be scrapped. Historians have already waited too long for their freedom. But even the best rules deserve reexamination more frequently than once every 25 or 35 years.

I would also note that PRIM&R embodies the system’s longstanding failure to consult “all stakeholders.” If PRIM&R is serious about that phrase, it should endorse the American Anthropological Association’s call for “a commission constituted specifically of social scientists (e.g., sociologists and the like), humanistic social researchers (e.g., cultural anthropologists and the like), and humanists (e.g., historians, legal scholars, and the like) … tasked with developing alternative guidance appropriate for their fields.”

Does PRIM&R Welcome History Exclusion?

In its response to the 2011 ANPRM, PRIM&R denied that an exclusion for “certain fields of study” was “even worth considering.”

Now, in comments on the NPRM, PRIM&R Executive Director Elisa Hurley writes, “some of the exclusions proposed in the NPRM will likely be widely welcomed, such as the explicit exclusion of journalism, oral history, biography, and historical scholarship activities.”

As Ellen Bresler Rockmore reminds us, passive constructions (“will … be widely welcomed”) can disguise meaning, and that is the case here; is Hurley among those who will welcome the change?

I hope so. But in any case, PRIM&R seems to be finally acknowledging the grievances of historians and journalists and the need for reform.

PRIM&R Meets with OIRA

Elisa Hurley, executive director of PRIM&R, reports that representatives of her organization met with officials from the federal Office of Information and Regulatory Affairs (OIRA) to discuss a potential notice of proposed rule making (NPRM).

[Elisa Hurley, “PRIM&R Meets with OMB to Offer Input on Proposed Changes to the ‘Common Rule,’” Ampersand, June 8, 2015.]

Recall that PRIM&R’s comments on the 2011 ANPRM were ignorant of the concerns of qualitative researchers, particularly historians, misleading about the responsibility of IRBs for bad consent forms, and contemptuous of the goal of efficiency.

I hope OIRA will hear from other parties as well.

University of Washington IRB Demanded Dangerous Consent Form

The recent Nature story on ethics consultancies includes an example of counterproductive interference by an intransigent IRB.

[Dolgin, Elie. “Human-Subjects Research: The Ethics Squad.” Nature 514, no. 7523 (October 21, 2014): 418–20. doi:10.1038/514418a.]

McCarthy's Mysterious Mythmaking

PRIM&R has launched "People & Perspectives (P&P)," described as a "digital story-telling library." The site features a blurb by Joan Rachlin, PRIM&R's soon-to-retire executive director, who calls it "an enduring and dynamic record of our historical antecedents, how and when we come together."

But is anyone going to vet the accuracy of stories posted on the site?

That question is raised by a 4-minute clip (taken from a much longer November 2013 interview) with Charlie McCarthy, director of the Office for Protection from Research Risks from 1978 to 1992.

I have not watched the full interview (not transcribed, and therefore a chore). But the four minutes and 12 seconds on "social-behavior research" is by itself a disturbing stew of faulty memory and misinformation.



Here are some of the key inaccuracies.

Exemptions Don't Come to PRIM&R's Mind

In October, PRIM&R hosted a webinar on "Protecting Human Subjects in Qualitative Research: Ethical Considerations for IRBs and Researchers," hosted by Julie Simpson, director of research integrity services at the University of New Hampshire.

A follow up question, just posted the PRIM&R Blog, suggests that Simpson is unfamiliar with the Common Rule:

AS: Under what circumstances might qualitative research not require IRB review?
JS: None come to mind at this time. If the activity is research and it involves human subjects, then it needs IRB review.

Of course, 45 CFR 46.101 lists several circumstances in which human subjects research does not require IRB review, some of them--particularly (b)(2) and (b)(3)--of enormous importance to qualitative researchers.

To be sure, OHRP is primarily responsible for discouraging institutions from recognizing the exemptions. But it is a pity to see PRIM&R spread such misinformation.

Law Professor Decries, Ponders IRB Variability

Christopher Robertson, associate professor at the James E. Rogers College of Law, University of Arizona, laments the variation in IRB policies and practices from one institution to another and sees it as an opportunity for research.

[Christopher Robertson, "Variability in Local IRB Regulation: A Gold Mine for Future Research," Bill of Health, November 24, 2012. http://blogs.law.harvard.edu/billofhealth/2012/11/24/variability-in-local-irb-regulation-a-gold-mine-for-future-research.]

PRIM&R: Efficiency Is Not an Ethical Value

Before moving on to other subjects, I wish to note one last passage in the PRIM&R response to the ANPRM:

The stated purpose of issuing this ANPRM is to make the process of human subjects protection more efficient. We fully recognize the potential benefits of accomplishing this goal. But we also want to note that efficiency itself is not a moral imperative or even an ethical value; human subjects protection should not be compromised by a desire for increased efficiency, a view we believe OHRP shares.

Really?

Is there no ethical problem when bright, curious undergraduates are forbidden from seeking answers to their questions? When a researcher abandons a project after months of inaction by the IRB? Or finds that by the time she gets approval, she has missed the chance to do her research? When IRB inefficiency delays medical research that can save lives?

I had thought that PRIM&R recognized justice as an ethical value. And justice delayed is justice denied.

PRIM&R: IRBs Don't Write Consent Forms

PRIM&R's ANPRM comments make the helpful concession that "consent forms have become longer, more complicated, more 'legalistic,' and less useful as sources of information to potential subjects." But PRIM&R does not wish its members blamed for this trend, and claims that IRBs do not write consent forms. While this may be true in a technical sense, I believe it fails to acknowledge IRBs' share of responsibility for the problem.

Is PRIM&R "Unaware" of Historians' Complaints?

PRIM&R has posted its ANPRM comments.

There are a number of interesting points, but let's start with the response to Question 25, about the possible exclusion of "certain fields of study" from the Common Rule.

Executive director Joan Rachlin writes,

Regarding the ANPRM’s question about whether the Common Rule should be revised to explicitly state that certain activities that have traditionally not been viewed as research (classics, history, languages, literature, and journalism, e.g.) are not covered (Q. 25), PRIM&R is unaware that the failure to exclude these fields from the Common Rule has ever been a problem for scholars in classics, or literature, etc., and therefore questions whether such a provision is even worth considering. That said, PRIM&R suggests that determinations regarding what is and is not subject to IRB review should be made on the basis of the specific research activity in question, and not on the basis of an investigator’s scholarly discipline. This would address some current inconsistencies regarding what type of inquiry gets reviewed.

"Classics, or literature, etc.?"

What's up with the "etc."? Has Rachlin not heard the complaints from historians, linguists, and journalists? Or has she heard the rumbling, but thinks that if she avoids writing "history," "language," or "journalism," those problems will go away?

History. It's the new bear taboo.

Compliance Administrator Wants ANPRM to Address Subparts

Writing for PRIM&R's blog, amp&rsand, Wendy Tate, assistant director of process improvement and compliance at the University of Arizona, complains that the ANPRM fails to address the subparts of 45 CFR 46.

[Wendy Tate, "What's Missing in the ANPRM?," amp&rsand, 5 October 2011.]

Harvard Law Fellow Tweets PRIM&R SBER

Michelle Meyer, an academic fellow at the Petrie-Flom Center for Health Law Policy, Biotechnology and Bioethics, Harvard Law School, is live tweeting the ongoing PRIM&R SBER conference in Boston.

To read her tweets, which so far include some especially interesting comments from Ezekiel Emanuel and C. K. Gunsalus, search the hash tag #primr_sber11.

[Many thanks to Dr. Meyer for alerting me.]

AAHRPP and PRIM&R Plan Conferences

Both the Association for the Accreditation of Human Research Protection Programs, Inc ., (AAHRPP) and Public Responsibility in Medicine and Research (PRIM&R) plan conferences next month. AAHRPP's conference, "Breaking Down Barriers, will be held in Washington, April 6-8, then PRIM&R meets in Boston, April 28-29, for a "Social, Behavioral & Educational Research Conference."

A Plea for "Networked Learning"

Alexander Halavais reports on a recent workshop sponsored by the Digital Media and Learning Research Hub, which brought together experts to discuss the challenges that IRBs pose to research in digital media.

Having agreed that IRB review sometimes produces unnecessary delays, particularly when multiple IRBs must sign off on a collaborative project, the workshop participants

found that while there might be some fairly intractable issues, as there are for any established institution, some of the difficulties that IRBs and investigators encountered were a result of reinventing the wheel locally, and a general lack of transparency in the process of approving human subjects research. The elements required to make good decisions on planned research tend to be obscure and unevenly distributed across IRBs. From shared vocabularies between IRBs and investigators, to knowledge of social computing contexts, to a clear understanding of the regulations and empirical evidence of risk, many of the elements that delay the approval of protocols and frustrate researchers and IRBs could be addressed if the information necessary was more widely accessible and easily discoverable.

Rather than encouraging the creation of national or other centralized IRBs, more awareness and transparency would allow local solutions to be shared widely. Essentially, this is a problem of networked learning: how is it that investigators, IRB members, and administrators can come quickly to terms with the best practices in DML research? Not surprisingly, we think digital media in some form can be helpful in that process of learning.

That is not an implausible idea. Plans for IRBs to share problems and solutions date back to the early 1970s, and they resulted in such institutions as PRIM&R and the journals, IRB: Ethics & Human Research and, more recently, the Journal of Empirical Research on Human Research Ethics. But these are fairly low-bandwidth channels: infrequent conferences and journal issues, with a few dozen sessions or articles per year, and devoted primarily to biomedical research. Hardly enough to generate a sustained discussion of an issue like social computing.

Alternatively, there exist online exchanges, like the IRB Forum. But these may lack the rigor of the journals. Rather than offering "empirical evidence of risk," as Halavais wants, they can amplify unrealistic fears. As Norman Bradburn testified before the National Bioethics Advisory Commission in 2000:

What is bothersome to me is that -- and the trend that I see in IRB's -- is that they are becoming more and more conservative, that is there is a kind of network at least in the ones that -- there is a kind of -- I do ont know what you call it -- ListServ kind of network that administrators of IRB's communicate with one another and they sort of say here is a new problem, how do you handle that, and then everybody sort of responds. And what happens is the most conservative view wins out because people see, oh, gee, they interpret it that way so maybe we better do it too. So over time I have seen things getting more and more restrictive . . .

May I suggest, then, that without proper supervision, digital media can be a liability rather than an asset. The challenge for Halavais and his colleagues is to build a conversation that combines immediacy and scholarly care.

PRIM&R Plans SBER Webinar

On February 25, PRIM&R will host a "webinar" entitled "New Solutions to Ongoing Problems When Reviewing Social, Behavioral, and Educational Research." The two faculty are Mary Marshall Clark, who successfully liberated oral history from IRB oversight at Columbia, and J. Michael Oakes, whose chapter in Institutional Review Board: Management and Function more or less claims that questionnaires can drive people to suicide. That's an interesting contrast, though not interesting enough for me to pay $175 to participate.

Zywicki, "Institutional Review Boards as Academic Bureaucracies"

My colleague Todd J. Zywicki, of Mason's law school, has posted on SSRN his contribution to the forthcoming Northwestern Law Review issue on IRBs: "Institutional Review Boards as Academic Bureaucracies: An Economic and Experiential Analysis," http://ssrn.com/abstract_id=983649

As stated in the abstract, the article "argues that the problem is that IRBs are fundamentally bureaucracies, and that this bureaucratic structure explains much of their frequent suboptimal decision-making. The poor performance of IRBs is thus not a consequence of those individuals who comprise it, but rather a reflection of their bureaucratic nature. The bureaucratic nature of IRBs appears to do nothing to improve the decisions that they make, while being the source of many of their problems."

Zywicki is right to note that people who gain money, power, and prestige from controlling other people's work have an incentive to define their duties broadly, even as they fret about researchers' conflicts of interest. Ironically, he seems only dimly aware of the extent to which administrators have seized power from researchers. He writes, "IRBs are fundamentally bureaucracies," not understanding that IRBs themselves are committees of researchers, and that the true bureaucracies are the university compliance offices. He notes, "there is reported to be a growing IRB conference circuit," suggesting his unfamiliarity with PRIM&R and its administrator-dominated conferences. And he makes no mention of the creation, in 1999, of the "Certified IRB Professional" as a new kind of administrator, someone who has staked a great deal on the maintenance, if not expansion, of IRBs' reach.

But buried in the essay is another explanation of the IRBs' expansion:

With respect to university bureaucracies such as IRBs, at least some of the growth in their internal administrative burden has been spurred by governmental regulations. In addition, the preoccupation of IRBs with paperwork and forms has been promoted by a regime of “fear” of governmental oversight, “[f]ear by the institution that it will be ‘out of compliance’ with one or more aspects of the paperwork, and so subject to penalty upon audit (be that by the NIH, the Office for Human Research Protection, the US Department of Agriculture, or whatever other organization is involved).”

How much of the growth of IRB staffs is due to the internal dynamic Zywicki stresses compared to the growing external threat of federal penalty? One way to find out would be to compare the growth of compliance regimes to major OHRP enforcement actions. If as Zywicki notes, the Northwestern University Office for the Protection of Research subjects "grew from two full-time professionals in the late 1990s to 25 professionals and an administrative staff of 20 last year," I'd like to know how much of that growth took place as a response to the suspension of sponsored research at Hopkins, Virginia Commonwealth, and other universities.

Without far more research on the actual workings of IRBs and compliance offices around the country, we can't test hypotheses such as Zywicki's.

A Trimmer PRIM&R

In my previous posting, I detailed the ways that the composition of PRIM&R's board and the operations of its conferences fail to match its stated mission, and I suggested what it would take for PRIM&R to live up to its claim to serve "the full array of individuals and organizations involved in biomedical and social science/behavioral/educational research." The other option, of course, would be for it to scale back its ambitions to serving merely "the full array of individuals and organizations involved in biomedical research," a noble task and one that its biomedical membership is far better equipped to handle.

PRIM&R's adopting this approach would mean recognizing that current regulations were written with biomedical research in mind and have never suited social science. It should therefore use its prestige to advocate broader exemptions from federal regulation for non-biomedical research. Such exemptions might exclude from IRB review:

* Research involving survey or interview procedures, where the subjects are legally competent, and where the investigator identifies himself/herself, and states that he/she is conducting a research survey or interview

* Research involving the observation (including observation by participants) of public behavior in places where there is no recognized expectation of privacy.

and

* Research involving the observation (including observation by participants) of public behavior in places where there is a recognized expectation of privacy, except where all of the following conditions exist:

(i) Observation are recorded in such a manner that the human subjects can be identified, directly or through identifiers linked to the subjects,

(ii) the observations recorded about the individual, if they became known outside the research, could reasonably place the subject at risk of criminal or civil liability or be damaging to the subject's financial standing or employability, and

(iii) the research deals with sensitive aspects of the subject's own behavior such as illegal conduct, drug use, sexual behavior, or use of alcohol.

I fear that Dr. Cohen will reject such proposed exemptions, taking them as evidence that I have "no interest in making IRB review work better for researchers, but only in eliminating IRB review for [my] research." Before he does so, let me point out that these exemptions are not my creations, but those of a joint subcommittee of the American Association of University Professors' Committee A on Academic Freedom and Tenure and Committee R on Government Relations. They were published as a report on Regulations Governing Research on Human Subjects, Academe, December 1981, 358-370.

A member of the subcommittee, and a signatory to the recommendations, was Sanford Chodosh, MD, the founder of PRIM&R.

PRIM&R's Public Responsibilities

Jeffrey Cohen writes, "I don't understand why you continue to criticize a program designed to explain and encourage the use of the flexibility available in the regulation and to promote communication and collaboration between IRBs and investigators to facilitate research." His comment shows that I have not made clear my concern with PRIM&R's constitution and operation. Let me suggest, therefore, that PRIM&R is a semi-official body that fails to conduct itself in a manner commensurate to its power and responsibility.

We first need to understand PRIM&R's special status as a chosen instrument of the federal government. Unlike scholarly associations, whose letters to OHRP get brushed off, PRIM&R has long functioned as an arm of OHRP and its predecessor, OPRR. For example,

• OHRP promotes and distributes PRIM&R's "Investigator 101" CD-ROM. Similarly, PRIM&R prepared the first edition of the Department of Health and Human Services' IRB Guidebook.
  
• Former OPRR head Charles McCarthy serves on the PRIM&R board; Cohen, a former OHRP official, co-organized the last conference; and current OHRP officials participate as PRIM&R conference faculty.
  
• OHRP exhibits at PRIM&R conferences, and OHRP officials, particularly Michael Carome, use PRIM&R conferences to offer guidance that then gets broadcast by IRB consultants, such as Dr. Cohen.

Given these connections, an IRB member or staffer could reasonably assume that following PRIM&R's guidance is a good way to avoid sanctions by OHRP. (Indeed, she would be foolish to assume otherwise.) Since it these staffers who then return to their institutions and impose conditions on research there, PRIM&R is a key link in the chain between federal power and the daily work of researchers.

PRIM&R itself recognizes this function. For example, the recent conference provided attendees with documents offering guidance on the regulatory definition of human subjects research and the proper application of the exemptions. Since providing just such guidance is one of the responsibilities of OHRP, the conference organizers must feel pretty confident that they have OHRP's blessing to take over this important role. (Note that unlike real OHRP guidance, PRIM&R's documents are not made public.)

If this is so, what must PRIM&R do to use its power wisely and justly?

1. Identify its constituents

PRIM&R's website states that "since 1974, PRIM&R has served the full array of individuals and organizations involved in biomedical and social science/behavioral/educational research." But what constitutes that full array? Is "social science/behavioral/educational research" one category or three? If the latter, what disciplines come under which category? And are there varieties of research (such as folklore, nonfiction writing, and law) that fit none of those categories, yet that face IRB review?

I would like to see PRIM&R list all the disciplines that come under review by IRBs whose members it trains. Then it could try to make some distinctions, for example, between disciplines that offer therapy or advocacy and those that do not; disciplines that study the body and those that do not, disciplines that use formal scientific protocols and those that do not; and the like. This would lead to a second task:

2. Include a range of disciplinary perspectives

As I mentioned earlier, PRIM&R's board of directors is dominated by researchers and administrators from hospitals and medical schools. They cannot be expected to be expert in the ethics and methodologies of the full range of disciplines in the behavioral sciences, social sciences, humanities, and professions, nor could any group of scholars drawn from any one field. If PRIM&R wants to offer sound guidance to researchers in all fields subject to IRB review, it should include them on every level, from conference panels to editorial boards to the board of directors itself. The goal should be that each field has the chance to shape any guidance that affects that field, so PRIM&R does not again, for example, offer a conference panel on oral history with no historians present.

Since the vast bulk of IRB review does concern biomedical research, I would not expect equal numbers for researchers in other fields. But I do think that folklorists should have as much power to shape PRIM&R's advice on folklore as physicians have to shape PRIM&R's advice on medical research. If this requires a complex committee structure, so be it.

3. Include a range of viewpoints

Prior to the recent conference, Dr. Cohen wrote me, "if you would like to participate in the conference, we'd be happy to work you in to the program (provided you have something constructive to say)." Now he writes, "I can only conclude that you have no interest in making IRB review work better for researchers, but only in eliminating IRB review for your research."

Does that mean that someone who believes in wider exemptions from review has nothing constructive to say, and therefore no place in PRIM&R? This would contradict Cohen's earlier pride in having invited Linda Shopes, who has advocated an oral-history exemption far longer and more effectively than I. And it would contradict the above-mentioned panels at the last conference, which seem to assume that the proper scope of exemptions has yet to be determined.

I suggest that PRIM&R invite participants based on their knowledge, experience, and ability to represent their discipline, not their adherence to a party line, and that it make public its criteria for choosing participants in all of its endeavors. Perhaps I am not the right person to represent my field at PRIM&R conferences, but Taylor Atkins sure as heck isn't either.

4. Ease participation in conferences

Now I get to Dr. Cohen's specific invitation for suggestions for future conferences. I have three:

a. Don't schedule the conference during exam week. Dr. Cohen informs me that many researchers declined his invitation to participate in the conference. My guess would be that a big part of their refusal resulted from the fact that the second week of May is exam week around the nation--not a problem for federal officials, administrators, and consultants, but a big one for active teaching faculty. As it is, I am surprised he got three researchers from the University of Minnesota; it was exam week there too. October and November are pretty busy with conferences as well for many scholars. I suggest February or March might be more fruitful.

b. Announce the conference. I'm not sure how I found out about the 2007 SBER conference, but I know it was only a few weeks before the conference, and that I did not hear about it through any of my usual reading as a historian, such as H-ORALHIST, the list for oral historians. An open call for participants, months in advance, sent to scholarly newsletters and e-mail lists might boost participation.

c. Pay the costs of researchers. Maybe PRIM&R already pays the travel costs of conference faculty, but if not, it should. Most university professors are lucky to get funding to travel to one or two conferences a year, and they want to use this to attend conferences within their own disciplines. With PRIM&R charging hundreds of IRB staffers up to $950 to attend, it should be able to subsidize a few dozen researchers.

These last three suggestions might bring a few more researchers to the next PRIM&R conference, but I think the real problem is far deeper than participation at conferences. If, as Cohen suggests, PRIM&R truly seeks "to promote communication and collaboration between IRBs and investigators to facilitate research," it will have to work much harder to include the many investigators it has so long neglected.

PRIM&R Finds Another Social Researcher

On April 13, I noted that the preliminary faculty list for this week's "Social, Behavioral, Educational Research Conference" included only one social researcher among twenty faculty. One of the conference's organizers, Jeffrey Cohen, replied that "the final list will have more social scientists." Well, the final list includes one more researcher: Steven Pennell, Survey Director, University of Michigan. Since the total list has been expanded to 33 faculty, this brings the representation of social researchers from five percent up to six percent. Many of the panels will feature no such representation at all.

The conference announcement asks, "Why is it often so difficult for IRBs and investigators to work together effectively when reviewing sophisticated, socially sensitive social/behavioral/educational research?" One answer might be PRIM&R's apparent belief that IRB administrators can learn to regulate research without hearing from researchers.