Macquarie's Respect for Expertise

A few weeks ago I critiqued "An inside-outsider’s view of Human Research Ethics Review," a blog post by Greg Downey of the Department of Anthropology at Macquarie University, Australia. Downey complained in a follow-up comment that I had "singled [him] out on [my] blog for derision," an unfair charge, given the effort I have put into deriding a wide assortment of scholars and public officials. But he took the bait, and in two follow-up posts, Dr. Zachary Schrag on ethics, IRB & ethnography and Some practical notes on ethics applications, he explains some of the ethics review process at Macquarie. Together with the "Human Ethics" page of the Macquarie Research Office, these postings provide a glimpse at a system that cares more about research ethics than regulatory compliance.

Particularly striking is Downey's description of who reviews applications:

A kind of departmental review does take place within the university-wide committee at Macquarie, as members of the committee are clustered so that color-coded sub-groups do the preliminary and most serious review of applications for which they have special expertise. If an application has to go to the whole committee (for example, research with children, medical procedures, Aboriginal Australian groups, or ethically challenging research tends to), we usually turn to the members of our committee who are best versed in the area of study. If we have a particularly difficult ones, we’ll consult with a faculty member outside the committee who has special experience.

The key words here are "special expertise," "best versed in the area of study," and "special experience." Someone at Macquarie has decided that having a nutritionist review oral history, or an oral historian review nutrition experiments, is not in the best interest of researcher or subject, but that having knowledgeable people review applications might make everyone happy. I am particularly impressed that the experts do the preliminary review, which under the American regulatory framework (I'm not sure about Australia) would mean giving them the power to provide exemptions or expedited approval.

As I have noted in comments on his postings, Downey has yet to persuade me that even this level of expert review is necessary for projects by trained researchers that only involve survey, interview, and observation research, and I remain attracted to the University of Pennsylvania system, under which researchers are certified and then, largely, left alone. But I thank Downey for introducing me to a university that is thinking hard and creatively about ethical review.

Northwestern IRB: Unsystematic Interviews Are Not Subject to Review

Today's New York Times features a story, "Criticism of a Gender Theory, and a Scientist Under Siege," about the case of J. Michael Bailey, Professor of Psychology, Northwestern University. Bailey's controversial book about identity. The book provoked several complaints, including the charge by "four of the transgender women who spoke to Dr. Bailey during his reporting for the book . . . that they had been used as research subjects without having given, or been asked to sign, written consent."

As reported by the Times, the case was investigated by Alice Domurat Dreger, Associate Professor of Clinical Medical Humanities & Bioethics at Northwestern, who has posted a draft article on the subject, "The Controversy Surrounding The Man Who Would Be Queen: A Case History of the Politics of Science, Identity, and Sex in the Internet Age," [PDF]

Dreger finds that Bailey did not commit serious ethical violations, nor did he violate the requirements for IRB review:

the kind of research that is subject to IRB oversight is significantly more limited than the regulatory definition of “human subject” implies. What is critical to understand here is that, in the federal regulations regarding human subjects research, research is defined very specifically as “a systematic investigation, including research development, testing and evaluation, designed to develop or contribute to generalizable knowledge” (United States Department of Health and Human Services, 2005, sect. 46.102, def. “b”). In other words, only research that is truly scientific in nature—that which is systematic and generalizable—is meant to be overseen by IRBs. Thus, a person might fit the U.S. federal definition of “human subject” in being a person from whom a researcher gains knowledge through interpersonal interaction, but if the way that the the knowledge she or he intends to gain is unlikely to be generalizable in the scientific sense, the research does not fall under the purview of the researcher’s IRB.

It is worth noting here, for purposes of illustration of what does and doesn’t count as IRB-qualified work, that I consulted with the Northwestern IRB to confirm that the interviews I have conducted for this particular project do not fall under the purview of Northwestern’s IRB. Although I have intentionally obtained data through interpersonal interaction, the interview work I have conducted for this historical project has been neither scientifically systematic nor generalizable. That is, I have not asked each subject a list of standardized questions—indeed, I typically enjoyed highly interactive conversations during interviews; I have not interviewed all of my subjects in the same way; I have negotiated with some of them to what extent I would protect their identities. This is a scholarly study, but not a systematic one in the scientific sense. Nor will the knowledge produced from this scholarly history be generalizable in the scientific sense. No one will be able to use this work to reasonably make any broad claims about transsexual women, sex researchers, or any other group.

When I put my methodology to the Northwestern IRB, the IRB agreed with me that my work on this project is not IRB-qualified, i.e., that, although I have obtained data from living persons via interactions with them, what I am doing here is neither systematic nor generalizable in the scientific sense.

Clearly Bailey's work hurt the feelings of some people he wrote about, but, as Dreger notes, "scholarship (like journalism) would come to a screeching halt if scholars were only ever able to write about people exactly according to how they wish to be portrayed." Indeed, that's what social scientists have been arguing for three decades.

Guidance Creep

I am often frustrated by the argument that federal regulations provide, in Jeffrey Cohen’s words, “sufficient flexibility for the efficient and appropriate review of minimal risk research.” While individual IRBs have much to answer for, federal regulators have, over the years, stripped them of a great deal of flexibility.

I recently came across a striking example of this. In 1983, Richard Louttit of the National Science Foundation, who had helped craft the list of exemptions encoded in 45 CFR 46.101, explained them as follows:

Much research of minimal risk was exempted from IRB review in order to reduce the IRB workload so that research involving ethical questions could get more than cursory review. But some institutions have decided that the IRB, or its chairperson, must review proposals to decide if they are exempt from review. If this seems contradictory, it is. And this was not envisioned by the staff group which worked out the exemptions.

[Richard T. Louttit, "Government Regulations: Do They Facilitate or Hinder Social and Behavioral Research?," in Joan E. Sieber, ed., NIH Readings on the Protection of Human Subjects in Behavioral and Social Science Research: Conference Proceedings and Background Papers (Frederick, Md: University Publications of America, 1984), 179.]

Yet in 1995, the Office for Protection from Research Risks adopted just that contradictory position:

Institutions should have a clear policy in place on who shall determine what research is exempt under .46.101(b). Those persons who have authority to make a determination of what research is exempt are expected to be well-acquainted with interpretation of the regulations and the exemptions. In addition, the institution should be prepared to reinforce and review, as necessary, the method of determining what is exempt. OPRR advises that investigators should not have the authority to make an independent determination that research involving human subjects is exempt and should be cautioned to check with the IRB or other designated authorities concerning the status of proposed research or changes in ongoing research.
(OPRR Reports, 95-02) [21 July 2014 updated link.]

The regime in place today is far more intrusive than the one worked out in 1981. Changes in the regulations themselves are part of the problem, but so are radical reinterpretations like the one above.

Study Finds IRBs Impose Inappropriate Forms and Guidelines

I thank Brad Gray for alerting me to:

Sarah Flicker et al., "Ethical Dilemmas in Community-Based Participatory Research: Recommendations for Institutional Review Boards," Journal of Urban Health 84 (July 2007); 478-493.

I do not have access to the full article, but here's the abstract:

National and international codes of research conduct have been established in most industrialized nations to ensure greater adherence to ethical research practices. Despite these safeguards, however, traditional research approaches often continue to stigmatize marginalized and vulnerable communities. Community-based participatory research (CBPR) has evolved as an effective new research paradigm that attempts to make research a more inclusive and democratic process by fostering the development of partnerships between communities and academics to address community-relevant research priorities. As such, it attempts to redress ethical concerns that have emerged out of more traditional paradigms. Nevertheless, new and emerging ethical dilemmas are commonly associated with CBPR and are rarely addressed in traditional ethical reviews. We conducted a content analysis of forms and guidelines commonly used by institutional review boards (IRBs) in the USA and research ethics boards (REBs) in Canada. Our intent was to see if the forms used by boards reflected common CBPR experience. We drew our sample from affiliated members of the US-based Association of Schools of Public Health and from Canadian universities that offered graduate public health training. This convenience sample (n = 30) was garnered from programs where application forms were available online for download between July and August, 2004. Results show that ethical review forms and guidelines overwhelmingly operate within a biomedical framework that rarely takes into account common CBPR experience. They are primarily focused on the principle of assessing risk to individuals and not to communities and continue to perpetuate the notion that the domain of “knowledge production” is the sole right of academic researchers. Consequently, IRBs and REBs may be unintentionally placing communities at risk by continuing to use procedures inappropriate or unsuitable for CBPR. IRB/REB procedures require a new framework more suitable for CBPR, and we propose alternative questions and procedures that may be utilized when assessing the ethical appropriateness of CBPR.

Incestuous Gay Monkey Sex

Scott Jaschik reports on the American Sociological Association annual meeting, "Who’s Afraid of Incestuous Gay Monkey Sex?," Inside Higher Ed, 14 August 2007:

Mary L. Gray, an anthropologist at Indiana University at Bloomington, described her work in graduate school, which raised all kinds of red flags with her IRB at the time: She wanted to study the way gay, lesbian, bisexual and transgender youth develop their identities in the rural Southeast, and she wanted to base her research on interviews with such youth, under the age of 18, without their parents’ knowledge. Her project, she said, “had every imaginable red flag.”

With some regrets, she won IRB support by appealing to prejudice many have of the rural South. Although she had no evidence to make this claim, she argued that the situation in the rural South is “so awful” for the young people she was studying that she couldn’t possibly approach their parents for consent. (Actually Gray believes that the situation for gay youth is more subtle and less uniform than she suggested, but she guessed it would work with the IRB, and it did.)

Because the IRB was — like most IRB’s — oriented around medical research, not social science, the focus was on potential harm that Gray could cause her research subjects in person. Gray reported that she received relatively little questioning or guidance from her IRB on one of her major areas of research: what the young people she studied wrote about themselves online. Gray developed her own ethics rules (she wrote to the subjects to ask permission), but she was struck by what was and wasn’t considered important by the IRB.

To the IRB, “distance read as objectivity” and so was by definition “good,” she said. Never mind that what her subjects shared about themselves online was as important as the thoughts they shared in person. This points to Gray’s broader critique of the IRB process. Social scientists frequently complain about IRB’s failing to understand their studies, but Gray suggested it was time to move beyond the idea of just adding more social scientists to the panel. Rather, she said it was time to question certain underlying assumptions of IRB’s and whether they even make sense for social science. It’s not that Gray doesn’t think there are ethical issues researchers must consider, but whether the medical model can ever work for projects that don’t follow the pattern of having a hypothesis designed to lead to the dispassionate creation of generalizable knowledge.

Gray said that “IRB fatigue” is discouraging researchers — especially graduate students — from even trying to get projects approved.

I can't say that I'd want a graduate student in any field asking minors about their sex lives without some kind of supervision. But it sounds as though UC San Diego's IRB lacked the expertise to give Gray meaningful guidance. That lack of expertise is built into the system of local IRB review, and it can produce decisions that are too lax as well as those that are too strict.

Evolving Research

Two recent blog postings raise the question of the awkward fit between IRBs' insistence on protocol review and research, such as ethnography and oral history, which begins with no set protocol.

University of Winnipeg professor of politics Christopher Leo poses and then answers positively the question, "Does the Ethics Bureaucracy Pose a Threat to Critical Research?" This provocative essay raises so many key questions that I plan to return to it in future postings. For now, note Leo's description about the evolving nature of his work:

Many researchers concerned with politics and policy stay in regular touch with politicians and public servants and, in the process, ask them questions the answers to which may well be used in future publications. That is an essential part of the research process because regular contact with well-informed people makes it possible for researchers to stay abreast of events and identify important issues as they arise.

So when does a query become a research question and a conversation an interview that requires ethics review? The guidelines are little help in answering that question, but, if we take them literally, they would appear to have taken from university researchers a right that every ordinary citizen enjoys, namely that of picking up the phone and talking to a politician or public servant without applying for bureaucratic permission to do so.

Meanwhile, over at Savage Minds, Alex Golub, assistant professor of anthropology at the University of Hawai’i Manoa (aka Rex), touches on the same question in the posting, "Using informed consent forms in fieldwork." He writes, "In some cases I interviewed people I’d known for years. I’d have breakfast or lunch with them and then schedule the official ‘interview’ for later on in the week."

IRBs that rigidly follow a biomedical model for ethics may insist that research protocols be spelled out in advance--even demanding sample questions. Such demands are inappropriate for the kind of work described by Leo and Golub: keeping in touch with knowledgeable people over a period of time.

An alternative appears in the University of Pennsylvania's Policy Regarding Human Subject Research in the Sociobehavioral Sciences. That policy accommodates such projects by freeing researchers from the requirement to submit "a fixed research protocol":

Evolving Research

Evolving research is a class of research in the sociobehavioral sciences in which the questions that are posed evolve in the course of investigation. An example is ethnography, where research questions may only be clarified after a period of observation and where current findings drive the next steps in the study. This class of research typically involves studying human behavior in non experimental settings, with or without active participation by the investigator; but it can also occur in more structured observational settings (e.g., oral histories, focus groups). In specific cases, such research does not pose more than minimal risk to human subjects and is considered to be “exempt from review,” as stated below. An approved mechanism is necessary for presenting to the IRB a research protocol that will evolve in the course of investigation. This policy institutes such a mechanism via certification.

4a. Research involving only non-interventionist observation of behavior occurring in public (including domains of the Internet clearly intended to be publicly accessible), for which no identifying information is recorded, is exempt from review.

4b. Investigators are allowed to use their certification, as per policy item 1, as a reference for describing evolving research activities to the IRB in lieu of a fixed research protocol.

This policy eliminates the need for investigators doing evolving research to spell out the details of a dynamic research protocol. The IRB can be assured that the research will be conducted in an ethically appropriate fashion, with full protection of human subjects, when certified investigators attest that their pre-registered research plan will be conducted within the ethical framework laid out in the training program for which they are certified.

In other words, if you have shown you know what you are doing, you don't have to get the IRB's approval for specific questions or topics.

I must note, however, that the Penn policy includes this disclaimer: "Note that different studies by the same investigator(s) must be submitted to the IRB as separate research protocols. These must not be viewed as a single study evolving from one investigation into another." I wonder what the Penn IRB would do with someone like Leo or Golub, who has the audacity to keep in touch with people for years.

How Oral History Really Works

In "'If I See Some of This in Writing, I’m Going to Shoot You': Reluctant Narrators, Taboo Topics, and the Ethical Dilemmas of the Oral Historian," Oral History Review 34 (2007): 71-93, Tracy E. K’Meyer and A. Glenn Crothers present some of the challenges faced by oral historians in determining how much deference to give to a narrator's wishes. They describe a series of interviews they conducted with Marguerite Davis Stewart, a World War II Red Cross veteran, who contacted the Oral History Center at the University of Louisville offering to tell her stories.

Over three months, the interviewers recorded 32 hours of conversation, and found themselves wrestling with a number of questions. For example, they had to decide how much they should credit Stewart's dubious claim that she didn't think about race, how hard to press her for details of important but sensitive topics like her divorce, and how seriously to take Stewart's jests about not wanting some stories to be recorded. And since Stewart was blind and confined to a wheelchair, the interviewers had to decide how much time they could devote to helping her in daily life, and when to call in a qualified social worker.

Though the article does not mention IRB review, it suggests the futility of such review in solving the real questions that are likely to confront oral history interviewers. It shows that the hard questions were not present at the start of the process, but only emerged well after the point that an IRB would have approved, modified, or rejected a proposal. And that the tough questions were highly specific to the narrator. Thus, K'Meyer and Crothers write,

Conflict arose when, over time, Stewart sought our commitment to write a book according to her vision and outline. By that point in the interview process it had become clear to us that there would not be sufficient documentary resources to supplement her oral history and support a book-length manuscript. More important, because of her resistance there were gaps in the story that could not be filled. In short, we explained to her on frequent occasions that we could not write her book. We did agree to fulfill the original goal, to help her record the story, and to put an edited form of the transcript into the library for public use, organized according to the themes and chapters she identified. In effect, we promised separate products: her story deposited in the library and our interpretations in our academic work.

Even the most aggressive IRBs have not--as far as I know--demanded to review oral histories one narrator at a time, so they could not police such idiosyncratic concerns.

Finally, the questions raised in this article do not have clear right or wrong answers. It would be dreadful if an IRB could forbid the research or punish the interviewers because its members did not like the choices the interviewers made.

James Weinstein's Anti-Intellectualism

In his contribution to the Northwestern symposium, "Institutional Review Boards and the Constitution," Professor James Weinstein defends the constitutionality of IRB review for both biomedical and non-biomedical research. Though he does not quite say that IRB review of journalism would be unconstitutional, he is clearly troubled by it, so he needs to distinguish journalism from the social sciences. He does so by denigrating scholarly research as largely irrelevant to democracy:

Although there is obviously a considerable area of overlap, social science research and journalism have distinct purposes and perform different societal functions. The primary purpose of research, at least at universities, is to discover knowledge both for its own sake and for the betterment of human kind, not to improve the practice of democracy by supplying the public with information to facilitate the “voting of wise decisions.” While some of this knowledge will facilitate public as well as private decisionmaking, much will not. And while academic researchers occasionally engage in research with the specific purpose of producing information to persuade others on matters of public concern, such ideological advocacy is not the primary ethos of academic research and, indeed, can be in tension with the primary academic goal of discovering truth regardless of its political or social implication. In contrast, a primary purpose of journalism is to inform people about matters of public concern and to act as a “watchdog” against governmental abuse and official malfeasance. Similarly, the function of the editorial side of journalism is precisely to influence public opinion.

This essential difference between social science research and journalism is reflected in the publications through which these two professions communicate with the public—scholarly journals and academic books versus newspapers and magazines. Scholarly publications are usually aimed at a narrow, specialized audience and address people in their professional capacity; journalistic media, in contrast, are typically aimed at a more general audience, often addressing people in their capacity as citizens (as well as consumers). Moreover, not only do general circulation newspapers and magazines usually contain an editorial page in which the publisher and editors try to persuade people on matters of public concern, these publications also often have an opinion page in which members of the public are invited to do the same. In contrast, while some scholarly publications publish editorials and even more commonly letters from scholars in response to an article or some academic issue, these publications do not generally solicit the views of the general public. Accordingly, although scholarly journals and books, on the one hand, and newspapers and magazines, on the other, both form part of the “structural skeleton that is necessary for public discourse to serve the constitutional value of democracy,” it is the newspaper and magazines that form the “backbone” of this structure. Scholarly publications, in contrast, contribute less central support for the structure (a “shin bone” perhaps, to continue Post’s orthopedic metaphor).

The Court might therefore take a more refined approach to a law that imposed IRB regulations directly upon social scientists at research institutions than it would to similar restrictions imposed on journalists. Borrowing a page from its defamation jurisprudence, the Court might hold that to the extent the interviews related to matters of public concern such as attitudes towards homosexuality, abortion, or the war in Iraq, the regulations could not be applied. With respect to interviews on subjects not of public concern, such as the language used by waiters and waitresses or whether people can match dogs to their owners, the Court might well apply a lesser degree of scrutiny. This distinction would reflect the more important role played by the press in our democracy. It would also take account of the related fact that unlike the typical journalistic interview or survey, many social science interviews and surveys will not contribute to democratic self-governance.

It is true that avoiding an overly-refined doctrine that is difficult to administer argues for deeming all interviewing techniques directed towards producing public information a unified medium warranting the same high level of First Amendment protection. Still, the lesser contribution that social science interviews generally make to the “constitutional value of democracy” suggests that regulations that burden these communications should trigger something less than the exacting scrutiny that would be applied if IRB regulations were applied to journalists’ communication with their sources.

The idea that the "typical journalistic interview or survey" involves great questions of war and peace and commerce and culture, while the typical social-science interview or survey involves dog owners, is, I suppose, a testable hypothesis, though not one that Weinstein tests by sampling news stories and journal articles. Absent such evidence, it is an anti-intellectual slur.

Weinstein concedes that "funding issues aside, it is possible that the Court would find IRB regulations unconstitutional as applied to research using only traditional interview techniques," especially if they were imposed on survey or interview research "on matters of public concern." His tolerance for IRBs must therefore rest on his belief that such research is so rare that it's OK if it is caught up in a system designed to protect human subjects from useless research.

I will let Dr. Woodrow Wilson reply:

There is the statesmanship of thought and there is the statesmanship of action. The student of political science must furnish the first, out of his full store of truth, discovered by patient inquiry, dispassionate exposition, fearless analysis, and frank inference. He must spread a dragnet for all the facts, and must then look upon them steadily and look upon them whole. It is only thus that he can enrich the thinking and clarify the vision of the statesman of action, who has no time for patient inquiry, who must be found in his facts before he can apply them in law and policy, who must have the stuff of truth for his conscience and his resolution to rely on. . .

The man who has the time, the discrimination, and the sagacity to collect and comprehend the principal facts and the man who must act upon them must draw near to one another and feel that they are engaged in a common enterprise. The student must look upon his studies more like a human being and a man of action, and the man of action must approach his conclusions more like a student.

[Woodrow Wilson, "The Law and the Facts: Presidential Address, Seventh Annual Meeting of the American Political Science Association," The American Political Science Review 5 (February 1911), 8.]

Symposium on Censorship and Institutional Review Boards

The long-awaited Northwestern University Law Review Symposium on Censorship and Institutional Review Boards has hit the Web. I have read and blogged about some of these articles in their SSRN incarnations, but I look forward to reading the rest.

Insider-Outsider-Down Under

David Hunter kindly alerted me to "An inside-outsider’s view of Human Research Ethics Review," posted on Culture Matters, a blog hosted by the Department of Anthropology at Macquarie University, Sydney, Australia. The posting is anonymous, but the author describes himself "as a sitting member of Macquarie University’s review board for human research," which I think identifies him as Greg Downey. Alas, another failed attempt at keeping an informant anonymous.

Downey's [?] essay is a rebuttal to Jack Katz, "Ethical Escape Routes for Underground Ethnographers," American Ethnologist 33 (2006): 499-506. In that essay, Katz argues that protocol-review, the basic tool of ethics committees, is inappropriate for ethnographic fieldwork because fieldwork is so unpredictable. As Katz puts it

when researchers participate in naturally occurring social life and write field notes on what they observe, they often encounter people and behavior they cannot anticipate. Indeed, one of the strongest reasons for conducting participant-observation research is the view that the current state of knowledge, as shaped by fixed-design research that prespecifies the kind of people to be studied and the ways to study them (sampling designs, formalized questions and protocols, and time- and space-delimited situations in which to observe), is artificial, a product not of the subjects’ social lives but of prejudice.

He also notes that some ethnographers draw from past experiences and observations of everyday life, neither of which can be reviewed by an ethics committee. He then suggests ways that researchers and universities might escape the regulatory boundaries that seem to require prior review of research.

Downey [?] seeks to rebut this argument by insisting that prior review can improve the ethical content of anthropological research. He writes,

The ethics review process should not be avoided, escaped, or ‘exempted’ away. Rather, ethics review boards can be educated about ethnographic research methods and encouraged to produce clear standards for our research. I worry that too many anthropologists inadvertently suggest that ‘ethics’ is a bureaucratic hoop, that the ‘politics of representation’ is a far more worthy consideration than the nuts and bolts of evaluating risk, minimizing dangers to participants (including researchers), balancing public interest against risks that can’t be eliminated, and thinking hard about our relationships to our subjects, our collaborators, the field, the public at large, our home institutions, and those who support our work.

This is unresponsive to Katz's critique. If anthropologists lack "clear standards for our research," by all means they should develop them, with or without the help of scholars in other fields. But I don't see how ethics committees can contribute to this effort by demanding from researchers that they get "preauthorization for observations and interviews," as Katz puts it. That's just a demand for information that doesn't exist.

Downey [?] also writes,

Katz’s suggestion that decisions be made public—for many reasons—seems to me an excellent one, but that can happen on the departmental level even without university boards being involved. That is, each student need not invent the application anew every time. The goal is not vacuous or self-righteous ‘boilerplate language’ for ethics applications, as one recent anthropology blogger suggested, but a legitimate attempt by the anthropology community to think about effective techniques for recurring issues such as oral informed consent, naturalistic observation in heavily trafficked settings, the use of photographs, the protection of populations under dangerous regimes, and the ethical requirements on those learning of illegal activity.

OK, so we have some movement toward compromise and consensus. I would like to suggest that if Downey [?] believes that departments are the appropriate organs to publicize ethics-committe success stories, the first department to do so should be the Department of Anthropology at Macquarie University. A listing of proposed ethnography projects and the improvements made to them by the Macquarie ethics committee could prove a model for researchers around the world.

Finally, I thank Downey [?] for drawing my attention to Australia's
National Statement on Ethical Conduct in Human Research. This document is so shocking that I will save comments on it until I have more time.

To the Historian All Men Are Dead

Blogger's note: Since I find myself in dialogue with a bioethicist working in the United Kingdom (see IRBs vs. Departmental Review and its many comments), now seems like a good time to present the views of Sir John Kaye, a British historian of the nineteenth century who used correspondence and interviews, as well as documents, in his work. I first read this passage as an impressionable college freshman, and it shaped my views of what historians do and why.

ZMS

Sir John Kaye, "Preface," 1870.

From Kaye's and Malleson's History of the Indian Mutiny of 1857-8 (1897-1898; reprint, Westport, Connecticut: Greenwood, 1971), vol. 2, xi-xiii.

Dealing with the large mass of facts, which are reproduced in the chapters now published, and in those which, though written, I have been compelled to reserve for future publication, I have consulted and collated vast piles of contemporary correspondence, and entered largely into communication, by personal intercourse or by letter, with men who have been individually connected with the events described. For every page published in this volume some ten pages have been written and compiled in aid of the narrative; and if I have failed in the one great object of my ambition, to tell the truth, without exaggeration on the one hand or reservation on the other, it has not been for want of earnest and laborious inquiry or of conscientious endeavour to lay before the public and honest exposition of the historical facts as they have been unfolded before me.

Still it is probable that the accuracy of some of the details in this volume, especially those of personal incident, may be questioned, perhaps contradicted, notwithstanding, I was about to say, all the care I have taken to investigate them, but I believe that I should rather say "by reason of that very care." Such questionings or contradictions should not be too readily accepted; for although the authority of the questioner may be good, there may be still better authority on the other side. I have often had to choose between very conflicting statements; and I have sometimes found my informants to be wrong, though apparently with the best opportunities of being right, and have been compelled to reject, as convincing proof, even the overwhelming assertion, "But, I was there." Men who are personally engaged in stirring events are often too much occupied to know what is going on beyond the little spot of ground which holds them at the time, and often from this restricted stand-point they see through a glass darkly. It is hard to disbelieve a man of honour when he tells you what he himself did; but every writer, long engaged in historical inquiry, has had before him instances in which men, after even a brief lapse of time, have confounded in their minds the thought of doing, or the intent to do, a certain thing, with the fact of having actually done it. Indeed, in the commonest affairs of daily life, we often find the intent mistaken for the act, in the retrospect.

The case of Captain Rosser's alleged offer to take a Squadron of Dragoons and a troop of Horse Artillery to Dehli on the night of the 10th of May . . . may be regarded as an instance of this confusion. I could cite other instances. One will suffice:--a military officer of high rank, of stainless honour, with a great historical reputation, invited me some years ago to meet him, for the express purpose of making to me a most important statement, with reference to one of the most interesting episodes of the Sipáhi War. The statement was a very striking one; and I was referred, in confirmation of it, to another officer, who has since become illustrious in our national history. Immediately on leaving my informant, I wrote down as nearly as possible his very words. It was not until after his death that I was able orally to consult the friend to whom he had referred me, as being personally cognisant of the alleged fact--the only witness, indeed, of the scene described. The answer was that he had heard the story before, but that nothing of the kind had ever happened. The asserted incident was one, as I ventured to tell the man who had described it to me at the time, that did not cast additional lustre on his reputation; and it would have been obvious, even if he had rejoiced in a less unblemished reputation, that it was not for self-glorification, but in obedience to an irrepressible desire to declare the truth, that he told me what afterwards appeared to be not an accomplished fact, but an intention unfulfilled. Experiences of this kind render the historical inquirer very sceptical even of information supposed to be "on the best possible authority." Truly, it is very disheartening to find that the nearer one approaches the fountain-head of truth, the further off we may find ourselves from it.

But, notwithstanding such discouraging instances of the difficulty of extracting the truth, even from the testimony of truthful men, who have been actors in the scenes to be described, I cannot but admit the general value of such testimony to the writer of contemporary history. And, indeed, there need be some advantages in writing of events still fresh in the memory of men to compensate for its manifest disadvantages. These disadvantages, however, ought always to be felt by the writer rather than by the reader. It has been often said to me, in reply to my inquiries, "Yes, it is perfectly true. But these men are still living, and the truth cannot be told." To this my answer has been: "To the historian all men are dead." If a writer of contemporary history is not prepared to treat the living and the dead alike--to speak as freely and as truthfully of the former as of the latter, with no more reservation in the one case than in the other--he has altogether mistaken his vocation, and should look for a subject in prehistoric times. There are some actors in the scenes here described of whom I do not know whether they be living or whether they be dead. Some have passed away from the sphere of worldly exploits whilst this volume has been slowly taking shape beneath my pen. But if this has in any way influenced the character of my writing, it has only been by imparting increased tenderness to my judgment of men who can no longer defend themselves or explain their conduct to the world. Even this offence, if it be one against historical truth, I am not conscious of having actually committed.

Study Finds IRBs Make Consent Forms Harder to Read

In "Human-Subjects Research: Trial and Error," (Nature, 2 August 2007), Heidi Ledford writes:

When [physician William] Burman, of the University of Colorado in Denver, joined in two studies run by the Tuberculosis Trials Consortium, he knew that the consent forms needed to cater to people with an eighth-grade reading level (comprehensible to an educated 13-year-old). The trials involved multiple institutions, and the forms were sent to 39 institutional review boards (IRBs) — committees designed to determine whether a proposed experiment is ethically sound. The final approvals came in 346 days later, but what the IRBs sent back, Burman found disturbing.

"The consent forms were longer. The language was more complex," Burman says. "And errors were inserted at a surprising frequency." In one case, a potential negative side effect of the treatment had been accidentally edited out. Burman responded to the problem as any researcher would: he studied it. He had an independent panel review the changes. The reviewers found that 85% of the changes did not affect the meaning of the consent forms, but that the average reading level had jumped from that of an eighth grader to that of a twelfth grader (around 17 years old)1. His results confirmed something he'd suspected for some time. "I started to think about what was happening and it just seemed like the system was flawed." It was time to change the system.

Though the article (and the accompanying editorial, "Board Games," does not mention non-biomedical research, it does highlight the problem of relying on local IRBs, which are essentially committees of amateurs, to handle specialized tasks like drafting consent forms and determining procedures for confidentiality. See In Search of Expertise.

IRBs vs. Departmental Review

In comments on this blog's introduction, bioethicist David Hunter of the University of Ulster asked me about my preferred alternative to IRB review, and I mentioned my hopes for departmental review (hopes shared by the AAUP). Lest our conversation get lost in the comments, I am moving it to this new posting:

DAVID HUNTER:

I'd disagree on departmental review being best for two reasons.

1. While a committee which has some knowledge and expertise in the area of the project, too much expertise and it becomes too close to the subject matter. This can mean that it misses significant ethical issues because they are standard practice within a specific discipline. To give one example, psychologists often want to give part of their students grade (10%) for being involved in their research. Most RECs I am involved in don't allow this practice because it is felt it is unduly coercive. I imagine if a REC/IRB was entirely composed of psychologists they may disagree.

2. It is important for a REC to be substantially independent from the researcher, but this doesn't happen in departmental review, instead the REC has an interest in the research being let to go ahead.

My university presently runs on a departmental review model, and while I can't name names I have personally seen examples of both of the above issues coming up.

I've written about these problems here:
Hunter, D. 'An alternative model for research ethics review at UK universities' Research Ethics Review. (2006) Vol 2, No 2, 47-51.
(Which unfortunately isn't available online)

and here: Hunter, D. 'Proportional Ethical Review and the Identification of Ethical Issues Journal of Medical Ethics. (2007);33:241-245.

I certainly agree with you that IRBs shouldn't be dominated by medics and medical concerns, they instead should have a wide range of representation. I'm inclined to think though that the baseline ethical issues are similar and while different rules may be appropriate for different disciplines they flow out of the same background.

In terms of examples here are a few, I can't be too specific with details for reasons of confidentiality.

1. Study of sexual attitudes in school children. Asked very probing questions as one might expect, but didn't intend to get parental consent to carry out the research a parallel can be found here: India Research Ethics Scandal: Students made guinea pigs in sex study
No consideration had been given to what might have been done if there was disclosure of harmful behaviour etc.

2. Historian was going to civil war stricken country to interview dissidents about the war, intended to publish identifying comments (without getting consent for this) which were likely to be highly critical of the current regime.

3. Social scientist wanted to understand children's attitudes towards a particular topic. As a blind so that the participant would not know the questions they wanted to answers to, they proposed to use the becks depression index. This contains questions about self harm, future worth and was potentially very distressing, not at all appropriate as a blind.

4. Student wished to conduct interviews with employees of a company on an issue that could significantly damage the companies profitability. No consideration was given to how to best report this information to minimise harm to the company.

I'm inclined to think that any sort of research involving humans can lead to harm whether that is physical, social, financial, psychological or so on. As such the benefits and the risks need to be balanced, and it needs to be considered how to minimise that harm. That I take it is the job of the researcher. However, having sat on RECs for a while it is a job that sometimes the researchers fail at spectacularly, then it becomes the job of the IRB/REC. The difficulty is how, without full review by a properly constituted REC, do you identify those applications that have serious ethical issues?

ZACHARY SCHRAG:

Thanks for these examples.

First, let me state that I am primarily interested in projects that fit Pattullo's proposal of 1979: “There should be no requirement for prior review of research utilizing legally competent subjects if that research involves neither deceit, nor intrusion upon the subject’s person, nor denial or withholding of accustomed or necessary resources.” Under this formula, the projects invovling children (who are not legally competent) and the project involving undergraduates (whose course credit is an accustomed or necessary resource) would still be subject to review.

That said, I have little confidence that IRBs are the right tool to review such research. As for child research, under U.S. regulations, and, I believe, the rules of most universities, the studies could be approved by three IRB members wholly lacking in expertise on child development. (The regulations encourage but do not require the inclusion of one or more experts when vulnerable populations are involved.) Were I the parent of a child involved in such studies (and I'm proud to say that both my children have furthered the cause of science by participating in language studies), I would greatly prefer that the protocols be reviewed not by a human subjects committee, but by a child subjects committee composed mostly or entirely of people expert in child research.

For the psychology course and the history project, the real question is whether a departmental committee can be trusted to enforce its own discipline's ethical code. The code of the British Psychological Society forbids pressuring students to participate in an experiment. And the ethical guidelines of the the Oral History Society require interviewers "to inform the interviewee of the arrangements to be made for the custody and preservation of the interview and accompanying material, both immediately and in the future, and to indicate any use to which the interview is likely to be put (for example research, education use, transcription, publication, broadcasting)." So yes, those sound like unethical projects.

Perhaps some departments would fail to correct these mistakes, just as some IRBs and RECs get them wrong. At some level this is an empirical question that cannot be answered due to the uniform imposition of IRB review. In the U.S., at least one university (the University of Illinois) had a system of departmental review in psychology that worked without complaint until it was crushed by federal regulation in 1981. With the federal government imposing the same rules nationwide, we can only guess about how well alternatives would work.

Moreover, departmental review would allow committees to bring in considerations unknown to more general ethics committees. For example, the British and American oral history codes require attention to preservation and access to recordings, something that and IRB/REC is unlikely to ask about.

I would also add that something close to departmental review is typical of the standard IRB, i.e., one in a hospital or medical school. It's true that the U.S. regulations require "at least one member whose primary concerns are in nonscientific areas" and "at least one member who is not otherwise affiliated with the institution and who is not part of the immediate family of a person who is affiliated with the institution." But the rest of the members can be biomedical researchers of one stripe or another. If that's good enough for the doctors, how about letting each social science discipline form an IRB of its members, with a community member and a non-researcher thrown in?

Still, if IRBs/RECs limited themselves to holding researchers up to the standards of the researchers' own academic discipline, I wouldn't be complaining.

Where we really disagree, then, is on project 4. You write, a "Student wished to conduct interviews with employees of a company on an issue that could significantly damage the company's profitability. No consideration was given to how to best report this information to minimise harm to the company."

That sounds a lot like this case:

Kobi Alexander's stellar business career began to unravel in early March with a call from a reporter asking why his stock options had often been granted at the bottom of sharp dips in the stock price of the telecom company he headed, Comverse Technology Inc.

According to an affidavit by a Federal Bureau of Investigation agent, unsealed in Brooklyn, N.Y., the call to a Comverse director set off a furious chain of events inside the company that culminated yesterday in criminal charges against Mr. Alexander and two other former executives. Federal authorities alleged the trio were key players in a decade-long fraudulent scheme to manipulate the company's stock options to enrich themselves and other employees.

After the March 3 phone call from a Wall Street Journal reporter, the FBI affidavit said, Mr. Alexander and the other two executives, former chief financial officer David Kreinberg and former senior general counsel William F. Sorin, attempted to hide the scheme. Their actions allegedly included lying to a company lawyer, misleading auditors and attempting to alter computer records to hide a secret options-related slush fund, originally nicknamed "I.M. Fanton." It wasn't until a dramatic series of confessions later in March, the affidavit said, that the executives admitted having backdated options. The trio resigned in May.

That's an excerpt from Charles Forelle and James Bandler, "Dating Game -- Stock-Options Criminal Charge: Slush Fund and Fake Employees," Wall Street Journal, 10 August 2006. As far as I can tell, Forelle and Bandler made no effort to minimize the harms to the companies they studied or the executives they interviewed. Their "Perfect Payday" series won the 2007 Pulitzer Prize for public service.

Your insistence that an interviewer minimize harm is a good example of an effort to impose medical ethics on non-medical research, and a good reason to get RECs away from social science.