Georg KoppenThe Tor network is popular among researchers, which is great. However, this can easily lead to multiple research groups scheduling their experiments for the same or an overlapping timeframe, potentially impacting their results. This document explains how we intend to deal with that sitatution, both to help researchers plan their experiments properly and to give users and relay operators the right expectations regarding ongoing network-wide experiments and their potential effects.
Our strategy is three-fold:
-
We won't isolate every experiment, even though we can think about short-term exclusivity of experiments if researchers give us a good explanation of why they need it.
-
To make experiments successful, given 1., we'll focus on our coordination role raising awareness of concurrent experiments.
-
We encourage researchers to run their experiment a couple of times to help rule out noise and other side-effects influencing their results.
The Tor network is changing daily and is used for a myriad of purposes which holds as well during the time experiments are run on it. To a large extent those changes are not under our or anybody's control. Hence, one needs to take that constraint into account when planning network experiments, as trying to just run one experiment we know about at any time does not help with the underlying dynamic considerably. This goes as well for experiments we plan to run on our own, e.g. during our performance improvements work.
To help researchers with their planning (and thus a successful experiment) we provide information about experiments already running (or which are about to run) on the Tor network. One way doing so is via our tor-relays@ mailing list where we encourage researchers to write to, announcing their experiments' start/end and getting feedback from the relay operators community. Additionally, we added a Network Experiments entry on our status page, showing not only whether experiments are ongoing at the moment but as well which exactly are currently running and were running in the past. Each experiment entry contains the dates when an experiment started/ended, so one can go back into the past and learn which dates correspond to each activity. Finally, we coordinate the scheduling of experiments directly with research groups reaching out to us.
While careful planning is a necessary requirement for a successful experiment it often does not help against unknowns that might be encountered in an environment like the Tor network. To guard against noise and other effects potentially spoiling the experiment's results we therefore recommend running the proposed experiment a couple of times. We'll help with the respective scheduling where needed.