Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T17:48:38Zhttps://gitlab.torproject.org/legacy/trac/-/issues/33297Tune Onionperf Nagios alerts2020-06-13T17:48:38ZAna CusturaTune Onionperf Nagios alertsEnsure we have sensible warning and critical levels, and that we rate limit notifications.Ensure we have sensible warning and critical levels, and that we rate limit notifications.https://gitlab.torproject.org/legacy/trac/-/issues/33296Add Prometheus Node Exporter checks to Nagios2020-06-13T17:48:37ZAna CusturaAdd Prometheus Node Exporter checks to NagiosIn addition to standard CPU/RAM/Disk monitoring this could also check tgen processes.In addition to standard CPU/RAM/Disk monitoring this could also check tgen processes.https://gitlab.torproject.org/legacy/trac/-/issues/33294Update existing Nagios plugin for OnionPerf monitoring2020-06-13T17:48:37ZAna CusturaUpdate existing Nagios plugin for OnionPerf monitoringThe plugin should alert if any of the following are true:
- The web server does not exist or does not have sane responses
- There are excessive failures in the logs
- The onion services used for measurement are not reachable
In additio...The plugin should alert if any of the following are true:
- The web server does not exist or does not have sane responses
- There are excessive failures in the logs
- The onion services used for measurement are not reachable
In addition, the plugin should use stem where appropriate.https://gitlab.torproject.org/legacy/trac/-/issues/33273Prop 313: 8.2. Analyse and Monitor IPv6 Stats2020-06-13T17:48:14ZteorProp 313: 8.2. Analyse and Monitor IPv6 StatsAs part of Sponsor 55, the Tor Metrics team will analyse the new IPv6 connection and bandwidth statistics.
We will use their analysis to provide IPv6 progress reports.
We might also discover some bugs in tor's IPv6 stats code during th...As part of Sponsor 55, the Tor Metrics team will analyse the new IPv6 connection and bandwidth statistics.
We will use their analysis to provide IPv6 progress reports.
We might also discover some bugs in tor's IPv6 stats code during the first analysis.
See proposal 313, section 8.2, metrics analysis part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n355
The definitions of these statistics are in proposal 313, sections 4 and 5:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n118https://gitlab.torproject.org/legacy/trac/-/issues/33269Prop 313: 8.1. Check IPv6 Relay Consensus Counts Script2020-06-13T17:48:14ZteorProp 313: 8.1. Check IPv6 Relay Consensus Counts ScriptReview the IPv6 relay count script from proposal 313 (#33262).
There isn't really much to review here. There are only 4 calculations. We've done counts like this before.
See proposal 313, section 8.1, metrics review part:
https://git...Review the IPv6 relay count script from proposal 313 (#33262).
There isn't really much to review here. There are only 4 calculations. We've done counts like this before.
See proposal 313, section 8.1, metrics review part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n337https://gitlab.torproject.org/legacy/trac/-/issues/33260Add option to filter graphed OnionPerf results by relay fingerprint2020-06-13T18:04:09ZKarsten LoesingAdd option to filter graphed OnionPerf results by relay fingerprintWe want to provide the option to filter OnionPerf measurements by relay fingerprint to see how overall performance would change when removing relays with certain properties. The idea is that the user provides a set of fingerprints, eithe...We want to provide the option to filter OnionPerf measurements by relay fingerprint to see how overall performance would change when removing relays with certain properties. The idea is that the user provides a set of fingerprints, either as command-line arguments or contained in a flat file, and we exclude all measurements with those relays in the path. Ideally, we would make it possible for different sets of excluded relays to be plotted as different CDFs in the same graph.https://gitlab.torproject.org/legacy/trac/-/issues/33259Store measurements in a local database to reduce plotting time2020-06-13T18:04:09ZKarsten LoesingStore measurements in a local database to reduce plotting timeWe want to add graphs to OnionPerf that compare larger numbers of measurements. We also plan to add more parameters to plot different subsets of measurements. Having a local database for storing measurements and decoupling the parsing st...We want to add graphs to OnionPerf that compare larger numbers of measurements. We also plan to add more parameters to plot different subsets of measurements. Having a local database for storing measurements and decoupling the parsing step from the plotting step might reduce overall plotting time. This will be particularly useful when we include graphs based on Tor descriptors like flags or bandwidth or consensus weight.
Part of this ticket will be to figure out which database to use. Maybe we need to experiment with SQLite to see if it's fast enough for this purpose or whether we need something like PostgreSQL here.https://gitlab.torproject.org/legacy/trac/-/issues/33256Update CDF-TTFB graph2020-06-13T18:04:06ZKarsten LoesingUpdate CDF-TTFB graphWe're planning to add the CDF-TTFB graph discussed in #33076 to OnionPerf, but it looks like OnionPerf already contains a very similar graph. We'll know for sure after resolving #33255. We'll probably want to make changes to that graph t...We're planning to add the CDF-TTFB graph discussed in #33076 to OnionPerf, but it looks like OnionPerf already contains a very similar graph. We'll know for sure after resolving #33255. We'll probably want to make changes to that graph to be more similar what we discussed in #33076. For example, we'll want to split observations based on start time to distinguish experiment time from the time before and after, and we'll want to add subplots for different facets of the data like source or public/onion server.https://gitlab.torproject.org/legacy/trac/-/issues/33255Review existing graphing code2020-06-13T18:10:35ZKarsten LoesingReview existing graphing codeWe're going to modify and/or extend the graphing code in OnionPerf. Therefore we first need to review the existing graphing code. In particular, we need to document:
- external dependencies like plotting libraries,
- internal interdep...We're going to modify and/or extend the graphing code in OnionPerf. Therefore we first need to review the existing graphing code. In particular, we need to document:
- external dependencies like plotting libraries,
- internal interdependencies with other OnionPerf code parts,
- user interface with possible parameters,
- input data requirements, and
- all produced output files.https://gitlab.torproject.org/legacy/trac/-/issues/33178Figure out specific baselines we are interested in from a network health pers...2020-06-13T18:10:34ZGeorg KoppenFigure out specific baselines we are interested in from a network health perspectiveIn #33176 we checked what metrics/growth stats we currently have, which we need and whether all of the are collected properly.
In this ticket we should figure out specific baselines for our favorite stats. meejah came up with some thing...In #33176 we checked what metrics/growth stats we currently have, which we need and whether all of the are collected properly.
In this ticket we should figure out specific baselines for our favorite stats. meejah came up with some things that were worth collecting/investigating:
* expected failure rate for ciruits
* what % of exits are not expected to establish circuits
There might be more. This is likely a parent ticket and we should file child ones for more specific items.https://gitlab.torproject.org/legacy/trac/-/issues/33176Check whether all of our growth stats we want are collected and accurate2021-07-05T16:09:12ZGeorg KoppenCheck whether all of our growth stats we want are collected and accurateWe should start writing down which growth stats we are interested in and check whether we have them + if so whether they are accurate. For context: That's actually for network health work preparing some anomaly analysis we want to do.We should start writing down which growth stats we are interested in and check whether we have them + if so whether they are accurate. For context: That's actually for network health work preparing some anomaly analysis we want to do.https://gitlab.torproject.org/legacy/trac/-/issues/33165Number of direct users mysteriously spikes in US & NL2020-06-13T17:48:13ZcypherpunksNumber of direct users mysteriously spikes in US & NL
![https://metrics.torproject.org/userstats-relay-country.png?start=2019-11-08&end=2020-02-06&country=us&events=off](https://metrics.torproject.org/userstats-relay-country.png?start=2019-11-08&end=2020-02-06&country=us&events=off)
![h...
![https://metrics.torproject.org/userstats-relay-country.png?start=2019-11-08&end=2020-02-06&country=us&events=off](https://metrics.torproject.org/userstats-relay-country.png?start=2019-11-08&end=2020-02-06&country=us&events=off)
![https://metrics.torproject.org/userstats-relay-country.png?start=2019-11-08&end=2020-02-06&country=nl&events=off](https://metrics.torproject.org/userstats-relay-country.png?start=2019-11-08&end=2020-02-06&country=nl&events=off)
Seems to follow the same pattern as well. I've checked the other top 10 countries but none seems to be affected by this.https://gitlab.torproject.org/legacy/trac/-/issues/33111upgrade all metrics hosts to Debian buster2020-06-13T18:10:32Zanarcatupgrade all metrics hosts to Debian busterWe (TPA) want to upgrade all Torproject.org machines from Debian
9/stretch ("oldstable") to 10/buster ("stable") by the time the
"oldstable" release becomes end of life (EOL). This is expected to
happen somewhere around july 2020, this s...We (TPA) want to upgrade all Torproject.org machines from Debian
9/stretch ("oldstable") to 10/buster ("stable") by the time the
"oldstable" release becomes end of life (EOL). This is expected to
happen somewhere around july 2020, this summer.
Your team is responsible for these machines relevant for this
transition, which are currently running Debian stretch:
* colchicifolium
* corsicum
* materculae
* omeiense
* ~~oo-hetzner-03~~
* orestis
Would you please review the changes expected to happen during the
upgrade? You can see a summary of major package changes here:
https://www.debian.org/releases/buster/amd64/release-notes/ch-whats-new.en.html#newdistro
and here:
https://help.torproject.org/tsa/howto/upgrades/buster/#Notable_changes
Let us know if you need an advance notice before we perform the upgrade,
and how long it should be. If you need assistance or coordination, also
do let us know.
Otherwise the upgrade can proceed at any time and will be performed
before summer.
(In this specific case, I'm wondering if there are some boxes that we should decomission. In particular, there seems to be a lot of onionoo boxes in the fleet right now, with more under way (#32268). What's the plan for those? Should we upgrade the older boxes or phase them out when the new cluster is online?)
Thanks!https://gitlab.torproject.org/legacy/trac/-/issues/33090Make all Descriptor implementors serializable2020-06-13T17:58:25ZirlMake all Descriptor implementors serializableFor most distributed/cloud data frameworks, it is necessary to serialize objects to pass them around the system.
This change should be as simple as having the Descriptor interface extend Serializable, and then adding a constant serial U...For most distributed/cloud data frameworks, it is necessary to serialize objects to pass them around the system.
This change should be as simple as having the Descriptor interface extend Serializable, and then adding a constant serial UID to every implementing class.
As we currently don't declare these as serializable, nothing is trying to serialize them, so this should be a low-risk change.
The only issue I can see happening is if there are any members of implementation classes that are not serializable, but the majority of things are afaict.https://gitlab.torproject.org/legacy/trac/-/issues/33077Graph results from the torflow to sbws transition2020-06-13T17:58:26ZMike PerryGraph results from the torflow to sbws transitionAfter we get some solid graph methodology that we like from #33076, we should use those graphs to closely compare the sbws consensus votes to TorFlow consensus votes.
After that, when we try to switch to sbws, we should get before and a...After we get some solid graph methodology that we like from #33076, we should use those graphs to closely compare the sbws consensus votes to TorFlow consensus votes.
After that, when we try to switch to sbws, we should get before and after graphs of onionperf data, like we decided on for #33076.https://gitlab.torproject.org/legacy/trac/-/issues/33076Graph consensus and vote information from Rob's experiments2022-03-04T12:51:36ZMike PerryGraph consensus and vote information from Rob's experimentsThis is a ticket for the work to graph the historical onionperf data from Rob's relay flooding experiment.
Some discussion threads:
https://lists.torproject.org/pipermail/tor-scaling/2019-December/000077.html
https://lists.torproject.or...This is a ticket for the work to graph the historical onionperf data from Rob's relay flooding experiment.
Some discussion threads:
https://lists.torproject.org/pipermail/tor-scaling/2019-December/000077.html
https://lists.torproject.org/pipermail/tor-scaling/2020-January/000081.html
Basically, we want to have a standard way to graph results from key metrics from before, during, and after the experiment.
In this case, we want CDF-TTFB, CDF-DL from onionperf results.
We also want CDF-Relay-Stream-Capacity and CDF-Relay-Utilization for the consensus, as well as from the votes, to see if the votes from TorFlow drastically differ from sbws during the experiment.
https://trac.torproject.org/projects/tor/wiki/org/roadmaps/CoreTor/PerformanceMetrics
**Update from June 10, 2020: We finished the CDF-TTFB and CDF-DL portions by adding these graphs to OnionPerf's visualize mode. The remaining parts are the CDF-Relay-* graphs that are based on consensuses and votes. Keep this in mind when reading comments up to June 10, 2020.**https://gitlab.torproject.org/legacy/trac/-/issues/33066metrics dirbytes graph has two "0 Gbit/s" labels on the y axis2020-06-13T18:15:32ZRoger Dingledinemetrics dirbytes graph has two "0 Gbit/s" labels on the y axishttps://metrics.torproject.org/dirbytes.png?start=2019-10-29&end=2020-01-27
right now has three labels on the y axis: "0 Gbit/s", "0 Gbit/s", and "1 Gbit/s". That sure is weird.
Maybe the middle one is actually 0.5, and it gets rounded ...https://metrics.torproject.org/dirbytes.png?start=2019-10-29&end=2020-01-27
right now has three labels on the y axis: "0 Gbit/s", "0 Gbit/s", and "1 Gbit/s". That sure is weird.
Maybe the middle one is actually 0.5, and it gets rounded to zero?
I guess all sorts of things are possible. :)https://gitlab.torproject.org/legacy/trac/-/issues/33061archived bandwidth scanner files lack explicit source attibution2020-06-13T17:52:28Zstarlightarchived bandwidth scanner files lack explicit source attibutionCurrent files in
https://collector.torproject.org/archive/relay-descriptors/bandwidths/
https://collector.torproject.org/recent/relay-descriptors/bandwidths/
lack indication of which bandwidth scanner generated them. (files archived f...Current files in
https://collector.torproject.org/archive/relay-descriptors/bandwidths/
https://collector.torproject.org/recent/relay-descriptors/bandwidths/
lack indication of which bandwidth scanner generated them. (files archived from Tom's collection are attributed)
Collect these files here and abandoned an attempt to fill a gap due to this issue. Ad-hoc logic to bin them may be possible but is not trivial. Can provide attribution by sha256 digest for most of them if the file naming is improved.
original ticket #21378https://gitlab.torproject.org/legacy/trac/-/issues/33010Monitor cloudflare captcha rate: do a periodic onionperf-like query to a clou...2020-06-13T17:56:14ZRoger DingledineMonitor cloudflare captcha rate: do a periodic onionperf-like query to a cloudflare-hosted static siteWe should track the rate that cloudflare gives captchas to Tor users over time.
My suggested way of doing that tracking is to sign up a very simple static webpage to be fronted by cloudflare, and then fetch it via Tor over time, and rec...We should track the rate that cloudflare gives captchas to Tor users over time.
My suggested way of doing that tracking is to sign up a very simple static webpage to be fronted by cloudflare, and then fetch it via Tor over time, and record and graph the rates of getting a captcha vs getting the real page.
The reason for the "simple static page" is to make it really easy to distinguish whether we're getting hit with a captcha. The "distinguishing one dynamic web page from another" challenge makes exitmap tricky in the general case, but we can remove that variable here.
One catch is that Cloudflare currently gives alt-svc headers in response to fetches from Tor addresses. So that means we need a web client that can follow alt-srv headers -- maybe we need a full Selenium like client?
Once we get the infrastructure set up, we would be smart to run a second one which is just wget or curl or lynx or something, i.e. which doesn't behave like Tor Browser, in order to be able to track the difference between how Cloudflare responds to Tor Browser vs other browsers.
I imagine that Cloudflare should be internally tracking how they're handling Tor requests, but having a public tracker (a) gives the data to everybody, and (b) helps Cloudflare have a second opinion in case their internal data diverges from the public version.
The Berkeley ICSI group did research that included this sort of check:
https://www.freehaven.net/anonbib/#differential-ndss2016
https://www.freehaven.net/anonbib/#exit-blocking2017
but what I have in mind here is essentially a simpler subset of this research, skipping the complicated part of "how do you tell what kind of response you got" and with an emphasis on automation and consistency.
There are two interesting metrics to track over time: one is the fraction of exit relays that are getting hit with captchas, and the other is the chance that a Tor client, choosing an exit relay in the normal weighted fashion, will get hit by a captcha.
Then there are other interesting patterns to look for, e.g. "are certain IP addresses punished consistently and others never punished, or is whether you get a captcha much more probabilistic and transient?" And does that pattern change over time?https://gitlab.torproject.org/legacy/trac/-/issues/33008Display a bridge's distribution bucket2020-06-13T18:08:28ZPhilipp Winterphw@torproject.orgDisplay a bridge's distribution bucketBridge operators often want to know what distribution bucket their bridge fell into. Since #29480, one can find out by inspecting our [archived bridge pool assignments](https://collector.torproject.org/recent/bridge-pool-assignments/) bu...Bridge operators often want to know what distribution bucket their bridge fell into. Since #29480, one can find out by inspecting our [archived bridge pool assignments](https://collector.torproject.org/recent/bridge-pool-assignments/) but that's cumbersome and not user friendly. We should instead show the bucket on the bridge's relay search page. How can we get this done?