The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2020-07-15T09:57:35Zhttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33974Update OnionPerf to TGen 1.0.02020-07-15T09:57:35ZKarsten LoesingUpdate OnionPerf to TGen 1.0.0[TGen 1.0.0](https://github.com/shadow/tgen/releases/tag/v1.0.0) comes with a "change in the format of some of the configuration options that breaks compatibility with the previous version 0.0.1."
I tried to update OnionPerf to write ou...[TGen 1.0.0](https://github.com/shadow/tgen/releases/tag/v1.0.0) comes with a "change in the format of some of the configuration options that breaks compatibility with the previous version 0.0.1."
I tried to update OnionPerf to write out TGen files that TGen 1.0.0 understands. Here's the diff:
```
diff --git a/onionperf/model.py b/onionperf/model.py
index 3c057c5..90c824e 100644
--- a/onionperf/model.py
+++ b/onionperf/model.py
@@ -77,9 +77,9 @@ class TorperfModel(GeneratableTGenModel):
if self.socksproxy is not None:
g.node["start"]["socksproxy"] = self.socksproxy
g.add_node("pause", time="5 minutes")
- g.add_node("transfer50k", type="get", protocol="tcp", size="50 KiB", timeout="295 seconds", stallout="300 seconds")
- g.add_node("transfer1m", type="get", protocol="tcp", size="1 MiB", timeout="1795 seconds", stallout="1800 seconds")
- g.add_node("transfer5m", type="get", protocol="tcp", size="5 MiB", timeout="3595 seconds", stallout="3600 seconds")
+ g.add_node("stream50k", recvsize="50 KiB", timeout="295 seconds", stallout="300 seconds")
+ g.add_node("stream1m", recvsize="1 MiB", timeout="1795 seconds", stallout="1800 seconds")
+ g.add_node("stream5m", recvsize="5 MiB", timeout="3595 seconds", stallout="3600 seconds")
g.add_edge("start", "pause")
@@ -88,9 +88,9 @@ class TorperfModel(GeneratableTGenModel):
g.add_edge("pause", "pause")
# these are chosen with weighted probability, change edge 'weight' attributes to adjust probability
- g.add_edge("pause", "transfer50k", weight="12.0")
- g.add_edge("pause", "transfer1m", weight="2.0")
- g.add_edge("pause", "transfer5m", weight="1.0")
+ g.add_edge("pause", "stream50k", weight="12.0")
+ g.add_edge("pause", "stream1m", weight="2.0")
+ g.add_edge("pause", "stream5m", weight="1.0")
return g
@@ -109,10 +109,10 @@ class OneshotModel(GeneratableTGenModel):
g.add_node("start", serverport=self.tgen_port, peers=server_str, loglevel="info", heartbeat="1 minute")
if self.socksproxy is not None:
g.node["start"]["socksproxy"] = self.socksproxy
- g.add_node("transfer5m", type="get", protocol="tcp", size="5 MiB", timeout="15 seconds", stallout="10 seconds")
+ g.add_node("stream5m", recvsize="5 MiB", timeout="15 seconds", stallout="10 seconds")
- g.add_edge("start", "transfer5m")
- g.add_edge("transfer5m", "start")
+ g.add_edge("start", "stream5m")
+ g.add_edge("stream5m", "start")
return g
```
I'll let an OnionPerf instance run for a day to see at the output, also to see if we need to make adjustments to OnionPerf's analyze mode due to slightly changed log messages.
Until then, do these changes above look reasonable? Or did I miss something? Thanks!OnionPerf: Scalability, Performance, Establishing Basline MetricsKarsten LoesingKarsten Loesinghttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33438OnionPerf: Scalability, Performance, Establishing Baseline Metrics2020-12-14T07:53:00ZGabagaba@torproject.orgOnionPerf: Scalability, Performance, Establishing Baseline MetricsIn this project, the Tor Metrics and Tor Network teams will work together to improve OnionPerf so that it is a useful tool for all developers and researchers. As a result of this project, we will be able to conduct more meaningful experi...In this project, the Tor Metrics and Tor Network teams will work together to improve OnionPerf so that it is a useful tool for all developers and researchers. As a result of this project, we will be able to conduct more meaningful experiments on the Tor network. Enhancing OnionPerf is a critical foundational step in the work to scale the Tor network.
The goals of this project are to:
* Make operational improvements to existing OnionPerf deployments and make it easier to deploy new OnionPerf instances;
* Expand the kinds of measurements OnionPerf can take by making improvements to its codebase; and
* Make improvements to the way we analyze performance metrics.
* [ ] [Objective 1: Make operational improvements to existing OnionPerf deployments and make it easier to deploy new OnionPerf instances.](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33320)
* [ ] [Objective 2: Expand the kinds of measurements OnionPerf can take by making improvements to its codebase.](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33322)
* [ ] [Objective 3: Make improvements to the way we analyze performance metrics.](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33326)
Teams involved:
network health team
network team
metrics team
research director
More information in https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor59OnionPerf: Scalability, Performance, Establishing Basline Metrics2020-12-01https://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33435Document BASETORRC environment variable2020-06-27T14:24:40ZAna CusturaDocument BASETORRC environment variableOP can configure the Tor client through the BASETORRC environment variable. This should be added to the documentation with examples.OP can configure the Tor client through the BASETORRC environment variable. This should be added to the documentation with examples.OnionPerf: Scalability, Performance, Establishing Basline MetricsPhilipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33434Allow users to select Onion Service version to measure2020-07-02T09:39:41ZAna CusturaAllow users to select Onion Service version to measureCurrently OP runs measurements using both v2 and v3 Onion Services by default, when the user may only wish to only test one of them.Currently OP runs measurements using both v2 and v3 Onion Services by default, when the user may only wish to only test one of them.OnionPerf: Scalability, Performance, Establishing Basline MetricsKarsten LoesingKarsten Loesinghttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33433Add error handling for older stem versions2020-06-27T14:24:41ZAna CusturaAdd error handling for older stem versionsCurrently OP fails if the python-stem version required for enabling v3 services is not found. Instead, it should warn the user and either continue with v2 services only, or exit and ask the user to manually exclude v3 services on the com...Currently OP fails if the python-stem version required for enabling v3 services is not found. Instead, it should warn the user and either continue with v2 services only, or exit and ask the user to manually exclude v3 services on the command line.OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33432Multiple downloads for oneshot mode2020-09-04T17:13:34ZAna CusturaMultiple downloads for oneshot modeCurrently, oneshot mode performs a singe 5M download and then exits. We should allow users to request multiple downloads, which will aid future OP testing setups.Currently, oneshot mode performs a singe 5M download and then exits. We should allow users to request multiple downloads, which will aid future OP testing setups.OnionPerf: Scalability, Performance, Establishing Basline MetricsPhilipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33422Include more events about CBT in results2020-06-27T14:24:41ZAna CusturaInclude more events about CBT in resultsOnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33420Add CBT events to Onionperf result files2023-01-23T14:49:54ZAna CusturaAdd CBT events to Onionperf result filesAs I understand this, the results should record whether the CBT was set at the time of measurement. This could just be a CBT_SET True/False field in the tpf results file. The CBT resets in certain conditions, and this should be reflected...As I understand this, the results should record whether the CBT was set at the time of measurement. This could just be a CBT_SET True/False field in the tpf results file. The CBT resets in certain conditions, and this should be reflected in the results. The events should be parsed from the tor logs.
We should check with Mike if this meets the requirements.
* [ ] Blocked by [tpo/core/tor#40002](https://gitlab.torproject.org/tpo/core/tor/-/issues/40002)OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33419Add analysis to support static guard measurements2020-06-27T14:24:41ZAna CusturaAdd analysis to support static guard measurementsOnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33399Measure static guard nodes with OnionPerf2020-10-09T16:08:16ZAna CusturaMeasure static guard nodes with OnionPerfThe specifications for measuring this are as follows:
- OP should measure one guard or set of guards at a time. It could use NumEntryGuards in the torrc file to support more than 1 guard
- A new guard/set of guards must be chosen after...The specifications for measuring this are as follows:
- OP should measure one guard or set of guards at a time. It could use NumEntryGuards in the torrc file to support more than 1 guard
- A new guard/set of guards must be chosen after a day of measurement
- All CBT data must be erased when choosing a new set of guards, after day of measurements (at UTC midnight). This could mean erasing or replacing the state file for CBT.OnionPerf: Scalability, Performance, Establishing Basline MetricsKarsten LoesingKarsten Loesinghttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33396Automatically compress Onionperf logs2020-07-02T09:40:48ZAna CusturaAutomatically compress Onionperf logsAutomatically compress Onionperf logs to save space. This could be done as part of the log rotation at midnight or separately.Automatically compress Onionperf logs to save space. This could be done as part of the log rotation at midnight or separately.OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33392Add new metadata fields to json output2023-01-23T14:48:37ZAna CusturaAdd new metadata fields to json outputAdd new metadata fields for experimental measurements to the onionperf json output.Add new metadata fields for experimental measurements to the onionperf json output.OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33391Add new metadata fields and definitions2023-01-23T14:48:40ZAna CusturaAdd new metadata fields and definitionsDefine the instance metadata fields to help us differentiate experimental measurements.Define the instance metadata fields to help us differentiate experimental measurements.OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33328O3.2 Include additional OnionPerf filters.2020-12-14T07:54:30ZGabagaba@torproject.orgO3.2 Include additional OnionPerf filters.We will add the ability to filter out arbitrary relays from arbitrary time periods of historical OnionPerf data and compare the performance metrics to the baseline for that period as CDF graphs.We will add the ability to filter out arbitrary relays from arbitrary time periods of historical OnionPerf data and compare the performance metrics to the baseline for that period as CDF graphs.OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33327O3.1 Develop developer-facing tooling to quickly graph baseline performance m...2020-12-14T07:54:26ZGabagaba@torproject.orgO3.1 Develop developer-facing tooling to quickly graph baseline performance metrics.In order for developers to evaluate performance metrics as we make network improvements, we will create developer tools that allow us to produce CDFs from snapshots of time for these OnionPerf metrics for periods of time that live networ...In order for developers to evaluate performance metrics as we make network improvements, we will create developer tools that allow us to produce CDFs from snapshots of time for these OnionPerf metrics for periods of time that live network experiments are running. These graphing tools will also work with OnionPerfs that are attached to the Tor network simulator Shadow,1 allowing us to compare the live network to the simulator.OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33326Objective 3: Make improvements to the way we analyze performance metrics.2020-12-14T07:54:22ZGabagaba@torproject.orgObjective 3: Make improvements to the way we analyze performance metrics.Under sponsor 59's project we work on a dev tool to help people to analyze performance metrics.
* [ ] [O3.1 Develop developer-facing tooling to quickly graph baseline performance metrics.](https://gitlab.torproject.org/tpo/metrics/onio...Under sponsor 59's project we work on a dev tool to help people to analyze performance metrics.
* [ ] [O3.1 Develop developer-facing tooling to quickly graph baseline performance metrics.](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33327)
* [ ] [O3.2 Include additional OnionPerf filters.](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33328)
More info: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor59OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33321O1.3 Make OnionPerf more accessible to researchers and developers2020-12-14T07:54:17ZGabagaba@torproject.orgO1.3 Make OnionPerf more accessible to researchers and developersIn order to gather sufficient network measurements and conduct a variety of experiments with multiple instances of OnionPerf, it must be straightforward and easy for researchers and developers outside of the Metrics team to spin up and m...In order to gather sufficient network measurements and conduct a variety of experiments with multiple instances of OnionPerf, it must be straightforward and easy for researchers and developers outside of the Metrics team to spin up and monitor their own OnionPerfs with different configurations. With the ability for more people to run more OnionPerf instances, we can perform more experiments from more vantage points. These experiments will not happen during this project, but are only possible with these improvements to OnionPerf.
* [ ] [Multiple downloads for oneshot mode](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33432)OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33320Objective 1: Make operational improvements to existing OnionPerf deployments ...2020-12-14T07:53:45ZGabagaba@torproject.orgObjective 1: Make operational improvements to existing OnionPerf deployments and make it easier to deploy new OnionPerf instancesUnder sponsor 59's project we will improve on how OnionPerf is being deployed.
More info: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor59
* [ ] [O1.1 Improve monitoring: We will produce a Nagios plugin for monitor...Under sponsor 59's project we will improve on how OnionPerf is being deployed.
More info: https://trac.torproject.org/projects/tor/wiki/org/sponsors/Sponsor59
* [ ] [O1.1 Improve monitoring: We will produce a Nagios plugin for monitoring OnionPerf instances to ensure that they are operating correctly.](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33318)
* [x] [O1.2 Improve ease of deployment and maintenance: We will produce Ansible tasks for deploying and managing deployments of OnionPerf instances, which also allow for performing upgrades and custom configuration changes.](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33319)
* [ ] [O1.3 Make OnionPerf more accessible to researchers and developers](https://gitlab.torproject.org/tpo/metrics/onionperf/-/issues/33321)OnionPerf: Scalability, Performance, Establishing Basline Metricshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33260Add option to filter graphed OnionPerf results by relay fingerprint2021-03-15T19:16:19ZKarsten LoesingAdd option to filter graphed OnionPerf results by relay fingerprintWe want to provide the option to filter OnionPerf measurements by relay fingerprint to see how overall performance would change when removing relays with certain properties. The idea is that the user provides a set of fingerprints, eithe...We want to provide the option to filter OnionPerf measurements by relay fingerprint to see how overall performance would change when removing relays with certain properties. The idea is that the user provides a set of fingerprints, either as command-line arguments or contained in a flat file, and we exclude all measurements with those relays in the path. Ideally, we would make it possible for different sets of excluded relays to be plotted as different CDFs in the same graph.OnionPerf: Scalability, Performance, Establishing Basline MetricsAna CusturaAna Custurahttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/33259Store measurements in a local database to reduce plotting time2023-01-23T14:48:44ZKarsten LoesingStore measurements in a local database to reduce plotting timeWe want to add graphs to OnionPerf that compare larger numbers of measurements. We also plan to add more parameters to plot different subsets of measurements. Having a local database for storing measurements and decoupling the parsing st...We want to add graphs to OnionPerf that compare larger numbers of measurements. We also plan to add more parameters to plot different subsets of measurements. Having a local database for storing measurements and decoupling the parsing step from the plotting step might reduce overall plotting time. This will be particularly useful when we include graphs based on Tor descriptors like flags or bandwidth or consensus weight.
Part of this ticket will be to figure out which database to use. Maybe we need to experiment with SQLite to see if it's fast enough for this purpose or whether we need something like PostgreSQL here.OnionPerf: Scalability, Performance, Establishing Basline Metrics