Chris Soghoian summarized McCoy et al and Castelluccia et al papers, and described some of the issues there. (Both involved collecting data from exit servers.) Relevant questions: Is it relevant whether research is specifically about Tor, or whether it's just a convenient point to eavesdrop on people? Is it relevant whether data is aggregated? Chris proposed: only eavesdrop a network in order to do research on that network. (IOW, no using anonymity nets as a source of eavesdroppable users who can't easily complain or TOS you..) Minimize collection or retention. Stuff should be legal in the country where it's performed. Vet your stuff with an IRB if you have one. He suggested that papers ought to need to include a standard section about what ethical issues there were and what the authors did about it. Chris's slides are attached at the bottom of this page.
Roger Dingledine spoke next. He said he'd raise lots of point and not answer them. He discussed whether legal standards were particularly relevant, claiming that legal protections fall short of security properties, but claiming that researchers need to hold to a higher standard than actual adversaries. He then talked about research groups who asked Roger about how to do such research, get referred to a lawyer, then give up, So conservative careful groups give up, and brash/reckless groups (nor roger's words) become the only ones to do collection.
Roger proposed: only study an anonymity network iin order to do research on that network. Minimize data collection. (But it's hard to know in advance what to minimize.) Only present data that relates to your hypothesis. (If your hypothesis is about whether an attack works on the network, don't add a section to your paper where you out users and explain where they live and what they're doing.) Don't keep data that isn't safe to publish. Explain data collection methodology in public before you actually do it, so that it can actually get reviewed. Reviewers should demand enough info in papers to know that data collection methodology was safe. [Roger mentioned a paper I talked with him about where the whole paper is about deanonymizing users using only a tiny little bit of data... and where the ethics section was just "We hashed IPs, so it's all okay."]
Roger talked about how this makes research harder. But we're privacy researchers! Shouldn't we be interested in the question of how to do privacy-preserving research? He asked another question: science! If you collect and discard data, how can your results be reproducible? How can others analyze your data?
His last point: Roger paralepticly said that he didn't say anything about the law. There are all kinds of bad laws, and all kinds of places where the law allows all kind of evil stuff. IRBs are a fine conecept, but they're mostly not set up to look over privacy issues.
Next was Damon McCoy, who talked about the long ago time when he did his 2008 study. He pointed out that his predecessor attack study on Freenet that he submitted in 2005 got no ethical issues raised by the reviewers at the time when he submitted it in 2005 or 2007 as a callow young grad student. Basically, the process had lots of opportunities to let him know that there were ethical issues involved before the gotterdamerung of PETS2008, and so they thought they'd done right by
He brought up risk v reward issues; pointed out that their IRB process retroactively reviewed their paper and concluded that it was exempt from IRB review. He talked for a while about.
Marcia Hoffman from the EFF talked a bit about US law, pointing out that even non-US researchers ought to know a bit about it. Generally, under the Wiretap Act and the PEN Register act, and the Stored Communications acts, observing traffic, or routing info, or disclosing stored info, are legally fraught. She suggested that any researchers involving observing network traffic (on Tor or not) ought to get run by a lawyer. She also pointed out that lots of good research will fall into the huge nebulous gray area in contemporary computer law, so that looking for an absolute "This is definitely legal!" go-ahead will scuttle fine research. So other things to ask are, "what will a judge think?" "what are the circumstances seem like?" and "will I look like a bad actor, or a negligent person?"
(And at this point I sadly closed the window and had to reproduce stuff from cut and paste, so I lost some really great audience discussion. :( )
Jean Camp argued that using Tor users as a research sample is bad science of the "because the light is better there". She pointed out that HHS is revising their ethical research for US medical research, and we should try to make sure they get data security right.
The guy behind me whose name I don't know pointed out that that the Tor part of the 2010 paper was a small part of the paper overall.
Alyssa (sp?) suggested that people should join their IRBs, that researchers should also talk to network admins who also think about it.
Tom Benjamin talked about IRB issues, and pointed out that IRBs for medical researchers in his experience encourage more disclosure of data than we'd consider reasonable in this community.
Caspar asked for summary.
Damon pointed out that attacks that don't get demonstrated don't get fixed.
Marcia asked everybody to ask for stuff about legal questions of the EFF or other such friendly lawyers.
Chris pointed out that automated attacks don't itself mean that stuff gets fixed. (qv firesheep)
Adam's proposed next steps:
- Improve this page as a record of the panel (all participants are welcome), but don't edit it into all things to all folks. The panel was useful, and we should have a record, rather than making this page a fill next steps thing.
- Create a list of next steps with people who want to ensure they happen. Examples:
- Create a PETS-IRB service
- Create a public document with known ethical failings and their solutions
- Evangelize the document to program committees so they're aware of it
- Engage in some ethical search engine optimization with titles, links, etc to help people find the doc
Here are some thoughts of Nick's own: They probably seem obvious.
- If you believe that people have any kind of moral right to privacy, then it's probably wrong to violate people's privacy without a very good reason, and without taking every effort to find an alternative.
- Laws are an inadequate guide. Researchers may IMO jaywalk or violate the Chinese firewall in good conscience. Contrariwise, researchers shouldn't run spamming botnets even in countries that lack laws against that.
- IRB in its conception is a good idea, but current IRB boards are not trained to evaluate the kind of research we do. We (the PETS community) know better than they about how data that seems to be minimized can pose a risk to users. That's one of the main threads of our field! Perhaps we ought to constitute ethical review bodies from our field.
- Running a packet sniffer is not leet; neither is running an exit node. Any research that consists only of those is not real novel science.
- Review is important as a principle; IRB is less so. Review matters because we are all our own most understanding judges, and we're all our own research's biggest fans: getting other people for perspective is the easy alternative to spending years in meditation to learn humility.
- Tor users (and users of other anonymity networks) can make easy victims. They're looking for privacy after all, and are unlikely to lose even more privacy by seeking redress. Using them as convenient targets is picking on the vulnerable because they won't fight back.