Sunday, May 8, 2016

Texas A&M IRB Imposed Review for Surveys of Public Officials

In their contribution to the PS symposium, Kenneth Meiera and Apolonia Calderona complain of IRB interference in work that is clearly exempt or not even human subjects research.

[Kenneth J. Meier and M. Apolonia Calderon, “Goal Displacement and the Protection of Human Subjects: The View from Public Administration,” PS: Political Science & Politics 49, no. 02 (April 2016): 294–98, doi:10.1017/S1049096516000238.]

Meier and Calderon complain of two types of IRB overreach. First, the Texas A&M IRB is reviewing activities that are not human subjects research:

Our surveys of school districts (Meier and Rutherford 2014) are required to go through the Texas A&M IRB process. These surveys ask for public information: What is the process for selecting members to the board (e.g., election procedures), what is the racial composition of the board, the student enrollment, and the district administrative and teaching staff? The surveys are not confidential; school districts are identified so that these data can be merged with other public datasets for analysis. The question posed to our IRB was: “Who are the human subjects in this research and what risks do they incur that we should be prepared to mitigate”? The answer was the people who filled out the form are human subjects and they might be subject to retaliation if it were revealed how they filled it out. Insofar as we do not know exactly who fills out this form (it is likely a staff person designated to respond to information requests), the idea that an employee would be subject to retaliation for providing public information—information often posted on the district website—strikes us as an argument that is absurd on its face.

Though the Meier and Rutherford article (n. 6) states that “all data and documentation for the replication of this work can be found at,” I was unable to find the survey questions for that article on the PERG site. But if Meier and Calderon are accurate in their description of the project, it does sounds as though the IRB is reviewing work beyond its authority.

Second, Meier and Calderon are angry that the IRB insists on reviewing surveys of public officials:

Public officials are specifically not covered by federal regulations for HSP [i.e., CFR § 46.101(b)3i]; clearly, this lack of coverage is consistent with the legislative history of HSP. The regulation states that research in which “the human subjects are elected or appointed public officials” are “exempt from this policy.” No one believes that the purpose of the policy is to protect the Public Health Service employees who designed the Tuskegee and Guatemala syphilis studies. As students of bureaucracy, we teach that bureaucrats are public officials subject to different laws and regulations for no other reason than that they work for the government. We could contend that, at a minimum, any appointed officials in a policy-making position (e.g., school superintendents, university presidents, and agency heads) are public officials and, therefore, research on them is not covered by federal regulation. The Texas A&M IRB rejected this position and insists that surveys of such officials are subject to review and regulation.

Given OHRP’s explicit guidance that school superintendents are public officials for purposes of the Common Rule, Meier and Calderon have a good case here. Less clear is whether the school principals they also survey count. OHRP has said that superintendents are public officials but teachers are not, so where does that leave principals?

Meier and Calderon might have more patience with their IRB if it gave good, or at least consistent, advice. Instead,

Our annual Texas Middle Managers Survey has the same research focus, design, and subjects every year because the goal is to study the management process over time. In every case in which we submitted the approved IRB application from the previous year, the IRB required additional changes. The variation both within and across universities prompts the question of how local IRBs implement national guidelines.

So much for Stark’s “local precedents.”

Frustrated, Meier and Calderon turn to their training to explain otherwise baffling behavior:

Goal displacement occurs when organizations lose sight of objectives (i.e., outcomes) and focus on procedures that may not be directly tied to achieving their goals. Goal displacement leads to suboptimization; that is, because IRBs maximize compliance with procedures, researchers spend more time on compliance procedures. Because human subjects’ risks vary greatly across research areas, projects that pose little risk have a higher than optimal cost–benefit ratio. Even the IRB suffers from goal displacement based on the medical–legal model because it spends more time than it should on low-risk projects; as a result, it has less time available for high-risk projects. This misallocation of resources then increases the possibility of an adverse event affecting human subjects.

Actually, it’s worse that that: IRB review may itself endanger research participants. As Meier and Calderon explain,

Several years ago, the Project for Equity, Representation, and Governance conducted a study on Latino high school dropouts. The study began with a large quantitative study of school districts to determine how well they were performing relative to their resources and constraints. The second part of the study selected 10 districts based on regression diagnostics— five districts doing exceptionally well and five doing poorly—for in-depth study. We promised confidentiality to the districts in return for access. The Texas A&M IRB requested the district names; we refused. We are confident in our own confidentiality procedures; the ability of our IRB to protect confidentiality was unknown and its implementation problems did not inspire optimism concerning its ability to protect confidentiality… .

[in footnote] After several rounds of negotiation and delays of months, the IRB agreed that if we provided a list of districts from which the 10 would be drawn, that would be sufficient. We provided a list of all 1,043 Texas school districts.

Reluctant to play such games forever, Meier and Calderon sketch out an alternative screening system in which the most intrusive research, involving the most vulnerable subjects, would get the most scrutiny. Of course, that is pretty much what was promised with the promulgation of the 1981 regulations, the basis for today’s Common Rule. And it’s terribly hard to know what intrusive, or risky, even means. So without specifying in painful detail that the IRB may not interfere with specific activities, it will be difficult to design a system that will defeat the “one-way ratchet” of ever more burdensome oversight.

No comments: