Sponsor 59

Project Title: OnionPerf: Scalability, Performance, Establishing Basline Metrics

Project Period: January, 2020 - December, 2020

Project Goals/Activities

In this project, the Tor Metrics and Tor Network teams will work together to improve OnionPerf so that it is a useful tool for all developers and researchers. As a result of this project, we will be able to conduct more meaningful experiments on the Tor network. Enhancing OnionPerf is a critical foundational step in the work to scale the Tor network.

The goals of this project are to:

Make operational improvements to existing OnionPerf deployments and make it easier to deploy new OnionPerf instances; 
Expand the kinds of measurements OnionPerf can take by making improvements to its codebase; and 
Make improvements to the way we analyze performance metrics.

Teams involved:

network health team
network team
metrics team
research director

Project Tracking

TicketQuery(sponsor=~Sponsor59-must,format=progress)

REVIEWS

JUNE'S PLAN

Tickets by objective and activity

Objective 1: Make operational improvements to existing OnionPerf deployments and make it easier to deploy new OnionPerf instances

==== O1.1 Improve monitoring: We will produce a Nagios plugin for monitoring OnionPerf instances to ensure that they are operating correctly.====

Nagios is an open source software that monitors systems, networks, and infrastructure. We need such a monitoring tool because when we deploy OnionPerf instances to run experiments, they typically run for days or weeks at a time. These experiments often require code changes to underlying Tor processes or to network-wide parameters, which makes them susceptible to breakage. When the instances break, our experiments are disrupted and thus ineffective. It’s cumbersome to manually check these instances for breakage or errors. Having robust monitoring tools makes it possible for our small team to run experiments efficiently.

O1.2 Improve ease of deployment and maintenance: We will produce Ansible tasks for deploying and managing deployments of OnionPerf instances, which also allow for performing upgrades and custom configuration changes.

Ansible is an open source software tool that automates and improves configuration management and application deployment. Without tools like Ansible, setting up or updating OnionPerf experiments is a fully manually process, which can be error-prone and cumbersome. When establishing multiple OnionPerf instances, automating the setup or update process with Ansible makes it possible to run these experiments efficiently and reproducibly. In this task we will also make software changes that solve an existing disk space issue on OnionPerf instances. Right now, an OnionPerf instance keeps writing logs to its disk until that disk gets full. To solve this problem, we will sync logs to another location and delete them after a short while to keep storage requirements for OnionPerf instances constant. Ideally, we would make these logs publicly available.

O1.3 Make OnionPerf more accessible to researchers and developers

In order to gather sufficient network measurements and conduct a variety of experiments with multiple instances of OnionPerf, it must be straightforward and easy for researchers and developers outside of the Metrics team to spin up and monitor their own OnionPerfs with different configurations. With the ability for more people to run more OnionPerf instances, we can perform more experiments from more vantage points. These experiments will not happen during this project, but are only possible with these improvements to OnionPerf.

Objective 2: Expand the kinds of measurements OnionPerf can take by making improvements to its codebase.

O2.1 Add instance metadata: We need a way to distinguish our current four long-term OnionPerf measurements that are automatically published to the Metrics portal (https://metrics.torproject.org) from short-term experimental measurements.

In this task, we will add instance metadata to OnionPerf’s results in order to differentiate each experiment; we will store that data along with the actual measurement data in a separate, single archive. Without this task, we will be unable to distinguish the new experiments from the currently running OnionPerf instances, which makes the data collected unusable.

O2.2 Develop at least one new OnionPerf model: An OnionPerf model defines the pattern of traffic that is generated by the instances for conducting a measurement.

For example, a bulk download model can be used to measure time to first byte and time to last byte simulating the download of a large file. In contrast, a ping/echo service model would be used to learn the round-trip time from a client sending a ping to receiving a pong back from a server, which more closely models a real-time chat application. These models allow us to evaluate distinct circumstances and collect distinct metrics, all of which are currently unavailable with the existing OnionPerf model. We will evaluate new OnionPerf models that will allow us to measure different network workloads and deploy at least one instance of these permanently. Deploying a new OnionPerf model allows us to gain new insight into the Tor network by taking measurements that are currently unavailable.

O2.3 Implement static guard node support for OnionPerf.

Tor clients generally make three-hop circuits (that is, paths that go through three relays). The first position in the path called the guard node and is always chosen as the first hop for a fixed period of time. Using static guard nodes help protect against a certain kind of anonymity-breaking attack. The long-running OnionPerf instances that are currently deployed do not implement this mitigation because choosing a static guard at setup time would heavily influence their measured performance. Instead, they’re configured not to choose a guard for each measurement. We want to be able to measure the effects of good, bad, or average guards and compare the results, which is not possible without the opportunity to select a static guard node for an OnionPerf instance. Implementing the option to use a static guard in future OnionPerf instances will allow us to measure Tor performance closer to how clients experience it.

Objective 3: Make improvements to the way we analyze performance metrics.

O3.1 Develop developer-facing tooling to quickly graph baseline performance metrics.

In order for developers to evaluate performance metrics as we make network improvements, we will create developer tools that allow us to produce CDFs from snapshots of time for these OnionPerf metrics for periods of time that live network experiments are running. These graphing tools will also work with OnionPerfs that are attached to the Tor network simulator Shadow,1 allowing us to compare the live network to the simulator.

O3.2 Include additional OnionPerf filters.

We will add the ability to filter out arbitrary relays from arbitrary time periods of historical OnionPerf data and compare the performance metrics to the baseline for that period as CDF graphs.

MUST Tickets without objectives

Any other possible ticket

Comments

Please register or sign in to add a comment.