Two newly PhD'd "cognitive neuroscientists"--Ogi Ogas and Sai Gaddam--got a book contract (rumored to be quite lucrative) with a popular press to write a book called "Rule 34: What Netporn Teaches Us About The Brain."
As part of their work, they launched an online survey aimed at authors of sexually explicit, online fan fiction. Many people who read the survey found it to be poorly designed and offensive, and anger grew as fan authors came to fear that the book would present erroneous information about their community.
The study was not IRB approved. Because the researchers had graduated from Boston University by the time they launched the survey, BU's IRB has disclaimed any authority over the matter, though it may have asked the researchers to stop using presenting themselves as being affiliated with the university. While some of the commentary on the event has included discussions about what the IRB might have done had it been presented the protocol, we can only speculate about whether IRB review would have changed the project for better, worse, or not at all.
Moreover, the chief concern of critics seems not to be that individual survey respondents would be harmed, but that their community as a whole would be harmed by a mass-market book written by inept, ignorant authors. Since the National Commission, policy makers have generally agreed that IRBs should not try to defend whole communities against mischaracterization by scholars.
Still, readers of this blog may be interested in a case where researchers' lack of preparation irreparably alienated the very people whom they wished to study.
For a good introduction, see Alison Macleod's human element blog. Many links follow.
Friday, September 4, 2009
Subscribe to:
Post Comments (Atom)
Posts
13 comments:
Hi there - just a quick comment. I followed this debacle 'live', so to speak, and read a great many comments on Ogi Ogas' blog as he replied to critics. I also tried taking the survey in 'real time', and it was beyond offensive.
It was so offensive that I initially wondered whether the whole thing was a flimsy guise for the collection of explicit personal information - it was rather like a market research call that descends into a dirty phone call.
While the writers expressed fundamental anxiety about the purpose of the research and how they would be portrayed, there was also plenty of disquiet about basic anonymity/confidentiality. These researchers also used a picture link from a Harry Potter film to attract attention from passing fans: there are very many Livejournal users under 18, and there was no warning upfront of quite the kind of adult detail being collected here.
In short, it was an utter mess, and it would have failed any set of research standards, let alone the IRB's.
I am active in LJ media fandom, and an academic who does fan studies. I've been following the whole debate closely, but I would say that beyond the idea of "they will stereotype the community unfairly," there were a number of specific human subjects concerns raised by those of us who are academics and who are trained in human subjects protections in different disciplines. These were communicated to Drs. Ogas and Saddam and ignored, except for Ogas' later accusation about "anonymous purported academics" trying to sabotage them.
I suppose the sabotage refers to queries sent to Boston University's IRB board by email and phone when the researchers refused to answer questions about IRB oversight regarding the major problems with their instrument (including the lack of warning against minors taking it, given that questions involved sexual imagery such as "do you have rape fantasies.).
A copy of the email I sent to the head and ass't head of the IRB committee, which I also copied to Dr. Ogas is in the next post because of comment length.
I am removing links that I sent because of moderation concerns and because many are now dead since Ogas shut down his posts.
I am using my real name and leaving my academic affilitation in the email (which I also sent to the head of my university IRB since I was in effect writing as a member of that committee).
Robin Anne Reid part 1
Dear Professor Berndt:
I am a professor in the Department of Literature and Languages, A&M-Commerce. As a new media and fan studies scholar, I serve as my department's IRB committee chair, and am a member of the IRB committee on campus. My area of IRB specialization is the issue of human protection standards in the emerging fields of internet research.
I have become aware of a "Fan Fiction Survey" being administered by Dr. Ogi Ogas and Dr. Sai Gaddam, Department of Cognitive and Neural Systems, Boston University. It may be that only Dr. Ogas is a faculty member; Dr. Gaddam is a recent graduate, according to his campus page.
BU WEB PAGE URLs DELETED; PAGES DELETED
They have circulated news of this survey widely among online fans and fan communities, on LiveJournal, and elsewhere.
I have not taken the survey, nor have I participated in any discussion on Dr. Ogas' LiveJournal (LJ) which he set up to accompany the survey. I have read the discussion on the LJ and will be linking to some threads below.
The survey site is here: DELETED CAN BE SUPPLIED UPON REQUEST
Robin Anne Reid part 2
What I have seen has caused me to have some concerns which echo those raised by a number of people posting on Dr. Ogas' LJ, especially those who are, like me, academics doing scholarship in a variety of disciplines and active members of fandom. Besides issues not relevant to the Institutional Review Board (the badly written questions, the complete lack of knowledge concerning fanfiction), I am most concerned the cavalier approach to questions concerning human subject projection in this project being expressed by the academics doing the research. .
I acknowledge that the survey is anonymous, containing no request for information, and I gather from discussions in Dr. Ogas' LJ that they plan to erase the data afterwards. I also understand that this project is for a popular book rather than an academic article or journal although that information was not made readily available to participants. However, neither of those factors excuse the lack of a clear statement of age restriction on the survey (no minors), nor the triggering nature of questions, some of which involve possible illegal activities.
The content of the questions (including questions about respondent's sexuality, desires, and fantasies--including on question about whether they have rape fantasies--and a question about illegal drug use) is problematic *especially* because Drs. Ogas and Gaddam have taken no steps to warn minors away from the survey. They say they do not plan to survey minors, but they are posting in places where minors under 18 do post fan fiction on the internet, and there is no warning/statement about the age limit.
Here is the LJ post which includes all the questions of the survey and some critiques (posted after extensive feedback given on an earlier post):
OGAS LJ LINK DELETED: POST CLOSED
One among many critiques of the methodology and lack of ethical scholarship was raised on this thread:
OGAS LJ LINK DELETED: POST CLOSED
By the strict letter of the law, Drs. Ogas and Gaddam may be acting appropriately in their research, but their lack of concern regarding the age of respondents, the apparent duplicity about the nature of the product of the research, and their interactions with the very fans they are asking to participate in their research would cause me concern were they faculty on my campus. There have been past abuses of human subjects principles in fan studies, and online/media fandom is as a group well aware of past history and well read in much of the published scholarship concerning fan culture and fans. I am concerned about possible damage both to academic scholarship on fandom(s) and to fans who may be harmed by this research, especially if sloppily done work is distributed in a popular book that has not been subjected to academic peer-review. Nothing Dr. Ogas has said in his LJ has even begun to address these concerns which is why I decided to contact the IRB.
I hope that you can advise Drs. Ogas and Gaddam as to how they could improve their project with regard to human subjects issues.
This post has been included in a linkspam roundup.
The community as a whole was concerned about mischaracterization, but many people, including me, worried about the consequences of sloppy or absent confidentiality, data security, and anonymity. The survey writers put all the burden of confidentiality on the respondents -- the only advice was to remove cookies. The survey FAQ did NOT warn against filling out the survey on shared machines/ or where users could be seen.
The survey asked many possibly damaging questions, including whether respondents had ever used marijuana or alcohol while reading fanfic; the former, of course, is a Federal crime.
Many of us worried about the security of the data; others worried about the failure to make any attempt to exclude minors from the survey, which asked some explicit questions. And finally, we were concerned because at least twice Dr. Ogas appeared to have connected people's LJ postings with their survey responses, which could only have involved IP logging; again, this breaches confidentiality.
I can give you links substantiating all of these complaints, if you like; it will take a little digging, but the data is available.
We are also outraged that there was no disclaimer for adult content and that it allowed people of any age to access and answer (imagine your 13 year old coming across the question "do you have rape fantasies?").
"Moreover, the chief concern of critics seems not to be that individual survey respondents would be harmed, but that their community as a whole would be harmed by a mass-market book written by inept, ignorant authors."
This summary of subject concerns is incorrect and misleading. It was widely-reported within the subject community and widely-believed by the study population that identifying material was being garnered at the survey site, both through IP Logging and internet "bots", that would allow Drs. Ogas and Gaddam to attach legal names to internet "nicks". These concerns were never addressed in any fashion by Drs. Ogas and Gaddam, nor was there any indication at any time from the researchers that the anonymity of the participants would be maintained. Since the absence of participant anonymity would constitute participant "outing", previously seen by this subculture to lead to loss of employment, lost of custody rights to children, and even litigation, survey participants were concerned, and many of them expressed concern with the eventual book publication rather than with the methodology of data collection.
It is very clear that this survey was not conducted using IRB guidelines or oversight: lack of explicit opt-out information, lack of gatekeeper protocols, and most troubling of all, potential inclusion of minors in sexually-explicit anecdotal data collection due to both the absence of an age statement requirement for the survey, and the solicitation for participants in internet venues known to be frequented by minors.
As a member of the community surveyed, I can assure you that many community concerns applied to individual harms, particularly given the nature of the survey. There were no informed consent procedures; dubious protections of anonymity and confidentiality; study questions specifically asked about illegal drug use and sexual practice, placing individuals at risk for harm if identities were revealed; there was no attempt to screen out minors, and the survey recruited from some on-line communities where minors were known to be active.
As someone who has been following this débacle attentively from the beginning, I can't help but feel that you're simplifying the reason for the fanfic-writers' outrage. Dr Ogas is on record as presenting himself as "a neuroscientist from Boston University", i.e. implying that his survey was for peer reviewed research. He also failed to inform people of his book contract - in other words, he lied to his potential subject, both in the form of direct lies as well as lies by omission.
Furthermore, there were no attempts at screening out under-18's from a survey that included questions about illegal drug use and sexual habits, yet Dr Ogas assured those who raised their voices in concern that he was "not doing research on minors". Most of the fury seemed to stem from the pure ineptness of the research, as well as the utter arrogance of Dr Ogas in failing to accept criticism.
I would also like to point out that the criticism of Ogi Ogas and Sai Gaddam does not just come from a social science/human subjects research viewpoint, but that there were very serious concerns with their basic theories themselves. At best, what they were proposing to do would be considered pseudoscience and at worst a fairytale.
A complete outsider's take on the "neuroscience" of the proposed research is here:
http://neurocritic.blogspot.com/2009/09/rule-34-what-netporn-tells-us-about.html
and my own critique is here:
http://neededalj.livejournal.com/940.html
If you honestly think that there's nothing which needed IRB oversight in a research survey which:
- was posted in a place with a high proportion of users under 18,
- solicited participation with a link graphic including Harry Potter and Buffy the Vampire Slayer characters,
- had no age warning or content warning for participants,
- had no warning that the survey results would remain visible on the computer used unless the cookie associated was deleted,
- may possibly have allowed linking by the researchers between the survey answer and LJ identity,
- ask questions about rape fantasies, drug and alcohol use, pornography use and masturbation habits,
then I can see why such a lot of people I know in research have problems with IRBs. Good Lord.
Thanks to everyone for all of these comments.
Ogas and Gaddam went wrong in many various ways, some of which I was not aware of before reading these comments. Still, I think it is worth distinguishing those misdeeds that threatened the privacy of research respondents from the pseudoscientific methodology that distressed folks like neededalj, and which struck me as the main theme in much of the commentary I had seen. I am heartened by the care with which many of these comments--particularly Professor Reid's letter to BU--make just this distinction.
The alternative is to try to make the IRB an all-purpose research police. For example, J. Michael Oakes has suggested that IRBs must watch out for surveys that "may inappropriately create a social stigma that affects the entire . . . community." [J. Michael Oakes, "Survey Research," in Robert J. Amdur and Elizabeth A. Bankert, Institutional Review Board: Management and Function (Sudbury, Mass.: Jones and Bartlett, 2002), 431.] An IRB that measures surveys by such a standard may indeed spare some people pain, but only at the expense of negating academic freedom.
Post a Comment