Draft TCPS Allows Critical Inquiry of Aboriginal Governments

In 2006, Canadian historian Nancy Janovicek complained that ethics polices designed to protect Aboriginal peoples could allow Aboriginal governments to silence their critics by denying researchers permission to speak with them.

A new draft revised version of Chapter 9 of the Tri-Council Policy Statement addresses that problem. While it calls for researchers to secure the permission of Aboriginal governments for most types of research, it recognizes that this may be inappropriate when those governments themselves are being critically examined:

Article 9.7.   Research that critically examines the conduct of public institutions or persons in authority may do so ethically, notwithstanding the usual requirement, in research involving Aboriginal peoples, of engaging representative leaders.

As I have written before, the draft TCPS is inconsistent in its respect for critical inquiry, and ethics committees may not give sufficient weight to the disclaimers like this one. But such statements do give researchers a foothold in arguing for the freedom of inquiry.

Brown Pledges IRB Reform

The Brown Daily Herald reports "a series of reforms" at Brown University, intended to "streamline Institutional Review Board procedures." (Sydney Ember, "Reform in the Works for Research Review Board," 13 November 2009.)

The process started in 2007, when Brown faculty complained that IRB operations were inhibiting research, especially by undergraduates. Brown's Research Advisory Board convened a four-person, ad hoc subcommittee to investigate. That subcommittee released a draft report, "Undergraduate Research in the Social Sciences and the Institutional Review Board at Brown" in January 2009.

The January 2009 Report

The draft report found:

Many faculty members from the social sciences report some aspect of the IRB to be or to have been a burden, and a significant fraction feel that IRB practices are having or have had some dampening effect on the quality or availability of undergraduate research opportunities. There is the widely held belief that many social science projects typically consisting of interviews or surveys, have a low intrinsic risk. Most faculty members, however, do recognize the existence of some risk, depending on the nature of the activity, and the need for some type of oversight. Many faculty members believe that the formal and rigidly structured IRB process is not an optimal way to oversee and regulate undergraduate work, due to variable levels of organization, knowledge, and professionalism among the undergraduates, and the special time constraints associated with the undergraduate senior year.

The subcommittee offered three policy options:

A. "Brown adopts IRB review as the standard procedure for undergraduate theses and non-classroom projects dealing with human subjects."

B. "Continue the current system, but clarify several points and communicate the policy more explicitly to avoid faculty and student confusion."

C. "Adopt and communicate a policy in which non-federally-funded undergraduate work is not subject to IRB review, but rather to some other educational and oversight system tailored for undergraduates, to be defined."

It recommended Option B "as the best near-term solution," while keeping C open as a longer-term option.

Brown's Faculty Executive Committee (FEC) received the draft report at its January 2009 meeting. According to the minutes of that meeting, "The FEC was disappointed that the report did not address some of the larger issues. It appears that the definition of research is getting broader so that the IRB has their hand in every aspect of research." As far as I can tell, the report has still not been finalized.

Changes Since January 2009

According to the Daily Herald story, the university's Research Protections Office (RPO) claims that it has implemented many of the recommendations in the report, primarily by updating its website. Since many of the pages on that website are undated, it's hard to know how many have been changed since the release of the draft report in January.

Some of the report's recommendations do seem to have been implemented. For example, the report called for the prominent placement of the information that faculty advisors get to decide whether an undergraduate project needs IRB review. That information is now indeed prominently displayed.

In contrast, the report specifically objected to the assertion that "obvious examples of dissemination [and therefore generalizability] are publication in a scholarly journal, presentation at a professional conference, or placement of a report in a library." It recommended the deletion of the references to conferences and libraries, but as of today, they remain on the RPO website. Given that this was one of the most concrete, immediate proposals of the subcommittee, I suggest that the Daily Herald may have been premature in its announcement of reform.

Erroneous Assertions

Not mentioned in the faculty report are three significant misstatements about federal regulations in the RPO's document, Frequently Asked Questions.

1. "Federal Regulations are clear that it is not up to the investigator alone to determine if a project is exempt."

Federal regulations specify no such thing, as recently reiterated by OHRP.

2. "In certain situations, all involving no more than minimal risk, the IRB can waive the requirement that you obtain the participant's signature on the consent form."

In fact, 45 CFR 46.117(c)(1) also allows IRBs to waive the signature requirement when "the only record linking the subject and the research would be the consent document and the principal risk would be potential harm resulting from a breach of confidentiality." This is true even when the risk is greater than minimal.

3. "As long as your research involves collecting data or information from or about living individuals, you need to have it reviewed by the IRB."

This would put reading a newspaper under the jurisdiction the IRB. Brown's RPO doesn't really believe this, but it hasn't been careful with its explanations.

Unfinished Business

It's great that a Brown faculty committee has taken a look at IRB operations, and that the administration has made some changes in response. But the report indeed failed to address some of the big issues at stake, and the administration has failed to implement some of the minor reforms suggested ten months ago. This case suggests the difficulty of restoring the principle of faculty governance when it comes to social science research.

Princeton IRB Delays Student Research

The Daily Princetonian reports a sociology major's difficulties getting IRB approval for her senior thesis on Brazilian immigrants' changing perceptions of gender roles.

"It's such a long process that it thwarts your field work efforts," [Christine] Vidmar said, noting that the review board does not meet to approve proposals during the summer. "I've been waiting since I got back to school. The first deadline that I could apply for was in October. It's November now, and I still can't officially go do my interviews."

. . .

Vidmar noted that a well-researched thesis may require up to a year of field work, adding that review board hurdles make it more challenging to complete sufficient research. "If you're a senior and you don't have a thesis chosen by the spring of junior year then you can't start field research until November or December of senior year, which is really late," she said. "You need to be in the field in order to know what questions you're going to ask, but in order to be in the field you need to have given the IRB your questions ahead of time."

As horror stories go, this one is mild. But consider the following:

• While details are lacking, Vidmar's proposed research sounds to be exempt under federal regulations; she's just interviewing adults about their perceptions of gender.
  
• Princeton demands full board review for "almost all proposals," offering expedited review only on "an exception basis."
  
• The IRB does not meet for three and a half months in the summer and requires proposals to be submitted two weeks in advance of the meeting. Hence, a student who misses the late-May deadline must wait almost four months until late September for review.
  

Put these together, and it seems that Princeton has built a substantial impediment to students who would like to interact with people as a capstone to their undergraduate training but are unable to write detailed research protocols six months in advance.

This is not to say that undergraduates should be sent into the field without training or supervision. But review by at the department level, as suggested by Felice Levine and Paula Skedsvold; subcommittee review, as practiced at Macquarie University; or researcher certification as permitted at the University of Pennsylvania, might well achieve the same or better levels of oversight as full-board review without delaying the work and discouraging the curiosity of a student researcher.

Former IRB Chair Decries Inconsistency

Jim Vander Putten, Associate Professor of Higher Education at the University of Arkansas-Little Rock, kindly alerted me to his essay, "Wanted: Consistency in Social and Behavioral Science Institutional Review Board Practices," Teachers College Record, 14 September 2009.

Vander Putten, who chaired his university's IRB for six years, complains that IRBs fail to make decisions consistently. He accuses them of both under- and over-protection, and then offers two suggestions for reform.

Under-Protection

Vander Putten's case for under-protection concerns two invitations to take online surveys, both of them aimed at university administrators and faculty. He is upset about their lack of informed-consent apparatus:

Although these were low-risk social and behavioral science studies, IRBs should not be absolved from ensuring that researchers will fully inform prospective participants about specifics of the research tasks. I deserve to know the answers to a number of questions before I decide whether to participate in a study: What is the nature of my involvement?  Will survey completion take 5 minutes of my time or 35 minutes? How (not just if) will my identity be protected and confidentiality maintained in this study? In a web-based survey, am I required to respond to each question before proceeding to the next? Perhaps most importantly, what will researchers do with my responses if I decide to cease participation before completing the research task? Will they be kept or discarded? Do I have a voice in the matter? As a result of the absence of this information, I declined to participate in both studies.

There are certainly some nasty online surveys out there, that fail to screen for minors or that ask highly personal questions midway through the survey. And when I am asked to participate in a survey, I often appreciate an estimate of the time it will take. But I can't share Vander Putten's outrage about the two invitations he received to participate in surveys specifically aimed at university professionals. Without any IRB intervention, he proved capable of deciding for himself not to answer the surveys.

Consider that Vander Putten's own e-mail to me failed to warn me how long it would take to read his essay (5 minutes of my time or 35 minutes?), whether I was required to read each paragraph before proceeding to the next, or how (not just if) would my identity be protected and confidentiality maintained should I reply to his message. Instead of infantalizing me with such precautions, Vander Putten respected me as an autonomous adult.

Over-Protection

The case for over-protection is stronger:

A few years ago, another faculty member and I conducted a qualitative research study at five different Doctoral/Research institutions in the Southeast. As a professional courtesy, we informed each institution of our plans to interview faculty and staff on their campuses, and noted that the study had already been approved by our institution’s IRB. I was surprised when each institution required us to submit 'Exempt From Full Board Review' IRB proposals for review and approval as a precursor to conducting the research on their campuses. With my knowledge and expertise as a sitting IRB Chair, I volunteered to complete the proposals to increase the likelihood of IRB approval upon first review.

You can imagine my surprise when several of the IRBs rejected the proposals on the basis of an inconsistent array of style issues, such as consent forms not cumulatively paginated (e.g., 1 of 3, 2 of 3, etc) and either written or not written in the past tense. The time delays associated with revision and re-submission of these IRB proposals (some of which were rejected a second time) were measured in months, and would have been even longer had we been required to complete each institution’s responsible conduct of research training program. These delays began a chain reaction of subsequent delays in data collection, research conference proposal submissions and presentations, and manuscript submissions for publication consideration. For untenured faculty, these delays can present formidable obstacles to meeting institutional expectations for scholarly productivity leading to tenure and promotion.

Multi-campus projects like this one and those conducted by J. Paul Grayson and Linda Thornton are particularly good at exposing the arbitrary nature of much IRB decision-making. When one IRB insists on the past tense and another forbids it, you know that at least one board, if not both, has no idea what makes for good informed consent, yet it is willing to impose its guesswork on researchers.

Reform

Vander Putten concludes with two suggestions for reform.

The first is an endorsement of the Illinois White Paper's call for a national clearinghouse for IRB best practices. He writes, "Based on my experiences as an IRB Chair, researcher, and research study participant, this would be a useful development and should include guidance on consistent expectations for the use of informed consent documents regardless of research risk, data collection method, or funding source to provide optimal protection for prospective research participants."

There are two parts to this recommendation. One is for a clearinghouse, which is all to the good. The second part is for specific practices, which he recommends using the same guesswork that produced the inconsistent responses to his study proposal. Just as one board guessed that consent documents should be written in the past tense and another guessed that they should be written in the present, Vander Putten guesses that that every study should use informed consent documents, as opposed to unscripted explanations, or simple reliance on the context of the study. Such universality produces bad results. I hope, for example, that he would not expect researchers to hand out informed consent documents to people being observed in a public place.

Best practices cannot be determined a priori; they need research. If Vander Putten is serious about the clearinghouse idea, he needs to hold off on specific prescriptions for IRBs until competing proposals have been gathered and compared.

Vander Putten's second recommendation calls for

the expansion of federal regulations requiring researchers to complete training in the responsible conduct of human participants research before conducting research. This expansion should include minimum requirements for researchers to actually implement the ethical practices regarding informed consent that they learned in their institution’s training program. In education terms, it is inappropriate to train researchers on the history of informed consent and methods to incorporate specific elements of informed consent into their research, and then decline to hold researchers accountable for doing so. If a specified set of minimum requirements are implemented nationwide, then IRB review would begin to approximate peer review systems that are the bedrock of scholarly quality and integrity.

Though Vander Putten begins his essay with several links to IRB-related reportage in the Chronicle of Higher Education and Inside Higher Ed, he apparently missed David Glenn's November 2008 story, Scholars Mull Rules for Training in Research Ethics," and my own reportage about the replies received in response to OHRP's call for comments. Those replies showed significant frustration, among institutions and individual researchers, about the quality of existing training programs, making a regulatory mandate seem unwise.

While well intended, both of Vander Putten's recommendations show a lack of familiarity with the existing debates over IRB inconsistency. More reading might produce more nuanced suggestions for reform.

AAHRPP Retreats from "Objective Data"

In July I posted my comments on AAHRPP's Proposed Revised Standards. At the time, I applauded Element I.5.B for insisting that "based on objective data, the Organization identifies strengths and weaknesses of the Human Research Protection Program, makes improvements, when necessary, and monitors the effectiveness of the improvements."

How disappointing, then, to find that the Final Revised Accreditation Standards omit the phrase about objective data. Are we to infer that AAHRPP considers objective data too difficult a standard, and wants institutions to base their programs on subjective impressions? Of course, most of the IRB regime is based on such guesswork, but I had thought that AAHRPP seeks to raise the level of IRB review.