Sociologists Sarah Babb, Lara Birk, and Luka Carfagna surveyed qualitative sociologists about their IRB experiences and heard many of the usual horror stories, from insistence on inappropriate consent forms to the dribbling out of concerns over several rounds of comments. Few of their respondents are happy with the present system, though getting the right people in key positions can help.
[Sarah Babb, Lara Birk, and Luka Carfagna, “Standard Bearers: Qualitative Sociologists’ Experiences with IRB Regulation,” American Sociologist, October 6, 2016, 1–17, doi:10.1007/s12108–016–9331-z. Note: I read a version of this article in manuscript and am so credited in the article.]
Standardization and arbitrariness can coexist
The authors begin with a puzzle:
Researchers … have raised concerns about two apparently contradictory problems. On the one hand many studies have observed that different local IRBs arrive at inconsistent decisions regarding the same research proposal, causing significant difficulties for biomedical researchers working across multiple institutions… . On the other hand, researchers in the social sciences and humanities, especially qualitative interview researchers, are more likely to charge IRBs with extending homogeneous, biomedical standards to different kinds of research . . . Why does standardization appear to be such a dominant theme in qualitative researchers’ experiences with their IRBs?
As the following page makes clear, this is not so difficult a conundrum. What researchers colloquially call the “IRB” is more properly understood as a human research protection program (HRPP) with two parts. First, the institutional review board itself, composed (in a university setting) mostly of faculty, and second, the IRB administrative staff that conducts the initial screening and sets up forms and procedures. The staff component imposes highly standardized rules, then feeds any procedures raising concerns to the IRB itself, which abandons standards.
It’s like going to the airport and making it through a list of security procedures—photo ID, boarding pass, shoes off, belt off, liquids in a clear bag, laptop out—only to board a plane whose destination will be chosen based on the pilots’ whim after they’ve taken off.
Few qualitative sociologists like the IRB process as it now exists
Babb et al. interviewed “26 sociologists at nine institutions of higher education in the Northeastern United States.” Though they included institutions whose sociology chairs offered “a range of reported IRB experiences—two more negative, three more positive, and three mixed,” only two of the 26 people they interviewed “believed that IRB review was both important and legitimate in its current form,” and one of those serves on an IRB. Of the remaining 24 respondents, six “felt that IRB regulation of sociological research was antithetical to professional norms and illegitimate” and 18 described “a conflicted sense of regret that a system designed to attend to an important issue was causing problems.”
Rejection isn’t the problem
IRB apologists occasionally point to a low rejection rate as a sign that the system works. This article reminds us that IRBs don’t need to reject a proposal formally to hinder research.
Michael, at State University remarked that “there’s a lot of work to do to get the IRB to decide they’re not all that interested in what you’re up to.” Completing an IRB protocol—an extended form describing a research project–can take a significant investment of time, even where the research is very low risk. However, the most common concerns about paperwork were related not to the burden of initial application, but rather to “serial reapplication”—that is to say, the submission of multiple versions of the same protocol before receiving approval. As one researcher put it, “they want a great deal of information on exactly what questions will be asked and procedures for maintaining confidentiality, and … they often then go back and ask for more information again” (George, Eastern University). Gabrielle, at State University, similarly recalled her experience:, “they’ll send back a thing ‘what about this, what about this, what about this’ and now, you know, they can’t possibly have a form that would imagine every possible situation… [I]t took a good 6 to 9 months of just trying to…come up with a protocol that would appease them.”
and
Several researchers reported that they had decided not to do specific projects in order to avoid the hassle of getting through IRB. One claimed that she had “sworn off of doing any human subjects research anymore… I’ve found the process so crazy-making that … I just don’t want to deal with it anymore” (Gabrielle, State University).
IRBs lack expertise
- They know less than researchers:
Penelope, at Public University, recalled the months it took to process a student proposal to study a social movement in a conflicted region in Turkey—a situation about which IRB decision-makers had neither knowledge nor appropriate precedents. As she concluded: “it’s a group of people trying to think through the repercussions of something that they probably have less familiarity with than the researcher does.”
- They don’t accept standard methods:
At Urban University, a graduate student studying an anti-administration student social movement at another university was told he needed permission from that university’s administration. Barbara, the student’s advisor, remarked that “if the people studying the labor movement had to get permission from the factory owners, then nobody would have ever studied the labor movement!” Another example was the universal requirement of de-identifying informants–even in journalistic-style research where informants agreed to be quoted and where the credibility of the research rested on the attribution of quotes to well-known public figures. Still another example was snowball sampling: two researchers from Eastern University reported that their IRB had prohibited such sampling and suggested instead recruitment techniques commonly used in biomedical research, such as posting flyers in public places.
- They fetishize written consent forms:
Thirteen of our 26 researchers mentioned the requirement of signed informed consent forms as an issue they had encountered. For example, Clara from Rural College wanted to interview adult passers-by at a conference convention booth. Clara’s IRB required that she obtain signed consent forms; however, she found that the consent document posed insurmountable obstacles. As she put it “…[y]ou’re asking them for all this personal information…and you want them to sign a form that says that they’re giving permission for this interaction, and as soon as you pull out the form, they say, ‘no I don’t have time.’” George at Eastern University had seen the problems IRB informed consent requirements posed for graduate students engaging in participant observation: “they somehow expect that everything will come to a halt while the graduate students pull out a form and explain to people they’re talking with what it means and get their approval.” Consent forms were seen as particularly problematic when they were long, legalistic, and difficult to read, as several of our respondents complained.
Researchers sometimes evade
In 2007, Mark Ashcraft and Jeremy Krause interviewed 100 researchers, of whom 19 reported collecting data without IRB approval. An even higher percentage (6 of the 26, or 23 percent) of the respondents here “reported that they had either evaded IRB rules themselves or recommended that their students do so.”
Clara from Rural College, for example, discovered that it was impossible to interview passers-by if she presented them with the lengthy informed consent form required by her IRB (see quote above). Upon finding that people were unwilling to talk to her as soon as she presented the form, she decided to dispense with it. As she put it, “I just decided that I could either do field research or I could work with the IRB, but I couldn’t do both.” George, who had mentored graduate students doing qualitative research that had encountered similar obstacles, felt that “graduate students really are faced … with a choice of … either conforming to these rules and … not being able to do participant observation the way it should be done, or ignoring the committee…” (George, Eastern University).
People matter
As much as we might wish for a magic structural fix to IRB issues, for now, the best hope is to fill the IRB with good people and, more importantly, to hire the right staff. Babb et al. describe two institutions that had generated horror stories in the past but now seem to be getting better:
At Urban University, professional IRB administration appeared to be working relatively well for qualitative researchers. Urban’s IRB was run by a dynamic administrator who was committed to meeting personally with all researchers before they submitted protocols and engaging in back-and-forth communication. Barbara described this administrator as an enormous improvement over her predecessor in the position: “prior to that it just felt a little bit more like a black box, you know the IRB has its own email address…now I know that when I send something to that address it’s [the administrator’s name]…before then, what happened at the other end [laughs] of that email address was much less clear.” Because she was open to personal communication, it was possible for researchers to engage with her directly, and explain their reasoning if they thought a mistake was being made. An Urban University scholar described one case (the factory study that ran into the problem of site permission, described above) in which such engagement led to the reversal of the administrator’s previous decision.
and
At Yankee College, the old IRB chair had recently been replaced by Sam, a qualitative sociologist who had assembled a new group of faculty IRB volunteers to improve and clarify rules and procedures, and who made an effort to communicate with researchers directly rather than bureaucratically.
At its best, then, the IRB system works as a government of men, not of laws. Can we do better?
Posts
No comments:
Post a Comment