Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T18:04:34Zhttps://gitlab.torproject.org/legacy/trac/-/issues/34191Combine multiple analysis files into single data set2020-06-13T18:04:34ZKarsten LoesingCombine multiple analysis files into single data setOnionPerf's visualize mode currently supports specifying multiple data sets to be graphed with different labels for each data set. However, for each data set it's only possible to specify exactly one analysis results file. If we ever wan...OnionPerf's visualize mode currently supports specifying multiple data sets to be graphed with different labels for each data set. However, for each data set it's only possible to specify exactly one analysis results file. If we ever want to graph a data set covering more than a single day and compare that to another data set, we cannot do that at the moment.
Here's a possible user interface extension to allow specifying one or more analysis results files for each data set:
```
diff --git a/onionperf/onionperf b/onionperf/onionperf
index cb1899c..957c6df 100755
--- a/onionperf/onionperf
+++ b/onionperf/onionperf
@@ -326,8 +326,9 @@ files generated by this script will be written""",
visualize_parser.set_defaults(func=visualize, formatter_class=my_formatter_class)
visualize_parser.add_argument('-d', '--data',
- help="""Append a PATH to a onionperf.analysis.json analysis results file, and a LABEL that we
- should use for the graph legend for this dataset""",
+ help="""Append a file or directory PATH to an onionperf.analysis.json
+ analysis results file or directory of such files, and a LABEL
+ that we should use for the graph legend for this dataset""",
metavar=("PATH", "LABEL"),
nargs=2,
required="True",
```
I didn't write any other code for this yet.Ana CusturaAna Custurahttps://gitlab.torproject.org/legacy/trac/-/issues/34078Fix compilation warnings with clang 10.0.02020-06-13T15:53:21ZNick MathewsonFix compilation warnings with clang 10.0.0Fedora 32 includes Clang 10.0.0, which is a bit more strict about switch-statement fall-through annotations, and about converting enums to bools. This causes us to hit compilation warnings on all supported Tor versions.
(See also #3407...Fedora 32 includes Clang 10.0.0, which is a bit more strict about switch-statement fall-through annotations, and about converting enums to bools. This causes us to hit compilation warnings on all supported Tor versions.
(See also #34077 for the GCC version of this)Tor: 0.4.4.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33619Resolve TROVE-2020-0042020-06-13T15:52:14ZNick MathewsonResolve TROVE-2020-004This is a remotely triggerable memory leak on relays and clients, found by tobias pulls.
The issue is that when circpad_setup_machine_on_circ() is reached with an inconsistent internal configuration, it fails to free an object that is r...This is a remotely triggerable memory leak on relays and clients, found by tobias pulls.
The issue is that when circpad_setup_machine_on_circ() is reached with an inconsistent internal configuration, it fails to free an object that is replaced. It logs a bug warning, but that isn't enough.
Tobias Pulls found that this code was actually reachable, though, and results in a memory leak.Tor: 0.4.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33256Update CDF-TTFB graph2020-06-13T18:04:06ZKarsten LoesingUpdate CDF-TTFB graphWe're planning to add the CDF-TTFB graph discussed in #33076 to OnionPerf, but it looks like OnionPerf already contains a very similar graph. We'll know for sure after resolving #33255. We'll probably want to make changes to that graph t...We're planning to add the CDF-TTFB graph discussed in #33076 to OnionPerf, but it looks like OnionPerf already contains a very similar graph. We'll know for sure after resolving #33255. We'll probably want to make changes to that graph to be more similar what we discussed in #33076. For example, we'll want to split observations based on start time to distinguish experiment time from the time before and after, and we'll want to add subplots for different facets of the data like source or public/onion server.https://gitlab.torproject.org/legacy/trac/-/issues/33226Prop 311: 5. Declare Support for Subprotocol Version "Relay=3"2020-07-02T19:46:44ZteorProp 311: 5. Declare Support for Subprotocol Version "Relay=3"This ticket depends on relay IPv6 extends in #33220.
We reserve Tor subprotocol "Relay=3" for tor versions where:
* relays may perform IPv6 extends, and
* bridges might not perform IPv6 extends,
as described in this proposal.
See p...This ticket depends on relay IPv6 extends in #33220.
We reserve Tor subprotocol "Relay=3" for tor versions where:
* relays may perform IPv6 extends, and
* bridges might not perform IPv6 extends,
as described in this proposal.
See proposal 311, section 5:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n601Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33206Parse recently added lines2020-06-13T17:58:26ZKarsten LoesingParse recently added linesWhen working on votes and bandwidth files for #33077 last week I realized that some recently added lines are not yet parsed. Here's a list of lines and contents that I found when attempting to parse all 2020-01 descriptos that we should ...When working on votes and bandwidth files for #33077 last week I realized that some recently added lines are not yet parsed. Here's a list of lines and contents that I found when attempting to parse all 2020-01 descriptos that we should parse:
- `"bandwidth-file-headers"` and `"bandwidth-file-digest"` lines in votes,
- bandwidth file digests (SHA256, base64-encoded) in parsed bandwidth file digests,
- `"bridge-distribution-request"` lines in bridge server descriptors, and
- `"fingerprint"` lines in bridge network statuses.
Once we have these we should consider having CollecTor log unrecognized lines, so that we learn about them sooner and without actively looking for them. That's not part of this ticket, though.
I already have the start of a patch and will continue working on that.Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/33191Write database tests for twisted adbapi for GetTor2020-06-21T18:06:06ZCecylia BocovichWrite database tests for twisted adbapi for GetTorWe discussed moving to a different sqlite3 api, but after talking with @meejah, it's a bad idea to mix and match twisted with other I/O operations.
So, we're opting to stay with twisted, write some tests that work, and see whether we ne...We discussed moving to a different sqlite3 api, but after talking with @meejah, it's a bad idea to mix and match twisted with other I/O operations.
So, we're opting to stay with twisted, write some tests that work, and see whether we need to make changes to how dbpool is called and used.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33159Write a proposal for monitoring IPv62020-06-13T15:50:42ZteorWrite a proposal for monitoring IPv6For sponsor 55, I need to write a proposal that covers these objectives:
* O1.4 - Measure the number of Tor relays that support IPv6 reachability checks (#33051)
* O1.5 - Measure the number of connections, and consumed bandwidth, using I...For sponsor 55, I need to write a proposal that covers these objectives:
* O1.4 - Measure the number of Tor relays that support IPv6 reachability checks (#33051)
* O1.5 - Measure the number of connections, and consumed bandwidth, using IPv4 and IPv6 (#33052)
My current thinking is that:
* tor should log the number and consensus weight of relays that support IPv6 reachability checks, because we will need those numbers during testing
* these numbers are available in the consensus
Here's what the proposal requires:
* calculate relay IPv6 reachability numbers a few times during the project (we may as well use the tor logs)
* collect IPv6 connection and bandwidth statistics on tor relays
* calculate the IPv6 connection and bandwidth amounts a few times during the project
Here are some other useful things we might do:
* split the collected IPv6 statistics by client/relay
* calculate the guard-but-not-exit relays that support IPv6 client connections
* log IPv6 statistics in tor's heartbeat logs (we can't use these logs for our project reports, because they only show the local relay's statistics)
* calculate IPv6 reachability relay count and consensus weight on consensus-health
* add a pseudo-flag for relay IPv6 reachability support in Relay Search
* add metrics graphs that shows our progress on
* IPv6 reachability
* client IPv6 support on relays
* IPv6 connections and bandwidth
We definitely won't have time to do all of these optional things, so we should priorise, once the essential work is done.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33029dir-auth: Dir auths should resume sending 503's but never to relays or other ...2020-11-13T13:39:33ZDavid Gouletdgoulet@torproject.orgdir-auth: Dir auths should resume sending 503's but never to relays or other dir authsThis is a child ticket from #33018.
The idea here is that even if we hit the global write limit (bw), we should not return 503 code but rather answer another directory authority.
Dirauth _must_ be able to talk to each other at all time...This is a child ticket from #33018.
The idea here is that even if we hit the global write limit (bw), we should not return 503 code but rather answer another directory authority.
Dirauth _must_ be able to talk to each other at all time regardless of the bandwidth state.
Setting 043 milestone because this should be considered a bug and could even be considered for backport since dirauth are suffering from this at the moment.Tor: 0.4.2.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/32992TBB Project for LZMA2020-06-16T01:10:45ZShane IsbellTBB Project for LZMACreate a tbb project to build source code targeting the android platform.
Source code: https://git.tukaani.org/xz.gitCreate a tbb project to build source code targeting the android platform.
Source code: https://git.tukaani.org/xz.githttps://gitlab.torproject.org/legacy/trac/-/issues/32991TBB Project For ZSTD2020-06-16T01:10:45ZShane IsbellTBB Project For ZSTDCreate a tbb project to build source code targeting the android platform.
Source is at
!https://github.com/facebook/zstd
Zstandard - Fast real-time compression algorithm. This is used to compress data sent over tor. This compression ...Create a tbb project to build source code targeting the android platform.
Source is at
!https://github.com/facebook/zstd
Zstandard - Fast real-time compression algorithm. This is used to compress data sent over tor. This compression is useful over slower mobile networks.
We will compile this into tor with _--enable-zstd_ flag.https://gitlab.torproject.org/legacy/trac/-/issues/32764Solve code issues that block running clang-format on our code.2020-06-13T15:49:10ZNick MathewsonSolve code issues that block running clang-format on our code.There are some aspects of our code that confuse clang-format, or produce bad output. In nearly all cases, this is a sign of bad code.There are some aspects of our code that confuse clang-format, or produce bad output. In nearly all cases, this is a sign of bad code.Tor: 0.4.3.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/32708manpage: alphabetize General Options2020-06-13T15:49:01ZTaylor Yumanpage: alphabetize General OptionsThis ticket is for Swati's alphabetizing change in #4310. Making this a child ticket so we can create other child tickets for working on alphabetizing the other sections.This ticket is for Swati's alphabetizing change in #4310. Making this a child ticket so we can create other child tickets for working on alphabetizing the other sections.Tor: 0.4.3.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/legacy/trac/-/issues/32576Fix race condition in snowflake broker2020-06-13T18:21:17ZCecylia BocovichFix race condition in snowflake brokerThere is a race condition with the snowflake heap that has been causing the broker to crash several times a day. This race condition has existed in the broker for several years, but some recent updates as well as the host migration manag...There is a race condition with the snowflake heap that has been causing the broker to crash several times a day. This race condition has existed in the broker for several years, but some recent updates as well as the host migration managed to shake it loose.
----
This race condition is causing the snowflake broker to crash repeatedly and often since the migration. We noticed because CollecTor stopped collecting metrics since the restart on 14 November 2019.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/32480Use Github/Gitlab releases to distribute gettor binaries2020-06-21T18:05:56ZCecylia BocovichUse Github/Gitlab releases to distribute gettor binariesRight now GetTor attempts to delete the previous git release branch and upload a new one due to [limits on repository sizes](https://help.github.com/en/github/managing-large-files/what-is-my-disk-quota#file-and-repository-size-limitation...Right now GetTor attempts to delete the previous git release branch and upload a new one due to [limits on repository sizes](https://help.github.com/en/github/managing-large-files/what-is-my-disk-quota#file-and-repository-size-limitations).
Perhaps a better way to handle this would be to set up [Github releases](https://help.github.com/en/github/administering-a-repository/about-releases). This can be done through the [REST API](https://developer.github.com/v3/repos/releases/).Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/32475Reduce the number of locales we provide updates for in nightly2020-06-16T01:09:38ZboklmReduce the number of locales we provide updates for in nightlyMaking updates available for each locale is costing time on nightly build/signing machines:
- generating the .mar files
- generating the incremental .mar files
- transferring the mar files between hosts to sign them and publish them
Wit...Making updates available for each locale is costing time on nightly build/signing machines:
- generating the .mar files
- generating the incremental .mar files
- transferring the mar files between hosts to sign them and publish them
With the current resources we have, I think it risks increasing the time to provide updates on nightly builds too much. I think we could start by providing updates for a subset of locales only, and think about increasing that list later.boklmboklmhttps://gitlab.torproject.org/legacy/trac/-/issues/32366doxygen: use directory hierarchy, and start documenting directories2020-06-13T15:47:46ZNick Mathewsondoxygen: use directory hierarchy, and start documenting directoriesInitial work here in branch `doxygen_org`, PR at https://github.com/torproject/tor/pull/1492 .Initial work here in branch `doxygen_org`, PR at https://github.com/torproject/tor/pull/1492 .Tor: 0.4.3.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/32209write description of config subsystem architecture2020-06-13T15:47:13ZTaylor Yuwrite description of config subsystem architectureTor: 0.4.3.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/32206write high-level outline of target modular tor architecture2020-06-13T15:47:12ZTaylor Yuwrite high-level outline of target modular tor architectureTor: 0.4.3.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/legacy/trac/-/issues/32173Update signing infrastructure to work with Apple's notarization process2020-06-16T01:08:37ZGeorg KoppenUpdate signing infrastructure to work with Apple's notarization processWe have some working way of dealing with Apple's notarization requirement. We should now update our signing infrastructure with what we've learned so far.We have some working way of dealing with Apple's notarization requirement. We should now update our signing infrastructure with what we've learned so far.Matthew FinkelMatthew Finkel