Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-15T23:01:26Zhttps://gitlab.torproject.org/legacy/trac/-/issues/34401Re-design Connect screen on Android2020-06-15T23:01:26ZMatthew FinkelRe-design Connect screen on AndroidFennec has an initial Connect screen (with a gear/cog for accessing Network Settings).
Maybe we skip this with #29590.Fennec has an initial Connect screen (with a gear/cog for accessing Network Settings).
Maybe we skip this with #29590.https://gitlab.torproject.org/legacy/trac/-/issues/34101Add tor-browser-build project for application-services2020-06-16T01:26:23ZGeorg KoppenAdd tor-browser-build project for application-servicesUnless we rip out all of the `application-services` (which I currently don't think we'll do) we need to have a project for it in `tor-browser-build`.
This will be a fun one to build. See the `lib` dir for some build scripts and [the meg...Unless we rip out all of the `application-services` (which I currently don't think we'll do) we need to have a project for it in `tor-browser-build`.
This will be a fun one to build. See the `lib` dir for some build scripts and [the megazord design](https://github.com/mozilla/application-services/blob/master/docs/design/megazords.md).Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/legacy/trac/-/issues/33889Functions with char* arguments are dangerous when used with casting2020-06-13T15:53:04ZGeorge KadianakisFunctions with char* arguments are dangerous when used with castingThere are various functions in our codebase which receive char* arguments when they actually receive a byte array. That's I guess because of the old C standard when uint8_t didn't really exist, so we had to use char* for everything.
Sti...There are various functions in our codebase which receive char* arguments when they actually receive a byte array. That's I guess because of the old C standard when uint8_t didn't really exist, so we had to use char* for everything.
Still, these days we use uint8_t* and this means that we need to cast it to char* when we use those functions. This can cause security issues because this casting is silencing type-safety warnings (as an example see https://github.com/torproject/tor/pull/1843#discussion_r406191281 from #33545).
For example, see how `fast_mem_is_zero()` is used 75% of the time with casting its first argument. Instead of doing this, we could make a new `fast_mem_is_zero_uint8t()` function that takes uint8_t as argument. Or even make a `fast_mem_is_zero_char()` function that takes char*.
In other functions, it might be possible to replace the `char*` with `void*` but in the case of `fast_mem_is_zero()` that's not possible because of the implementation of the function.
Other potential problem functions: `base64_encode()` `hex_str()` `crypto_rand()` `crypto_digest()`.Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33273Prop 313: 8.2. Analyse and Monitor IPv6 Stats2020-06-13T17:48:14ZteorProp 313: 8.2. Analyse and Monitor IPv6 StatsAs part of Sponsor 55, the Tor Metrics team will analyse the new IPv6 connection and bandwidth statistics.
We will use their analysis to provide IPv6 progress reports.
We might also discover some bugs in tor's IPv6 stats code during th...As part of Sponsor 55, the Tor Metrics team will analyse the new IPv6 connection and bandwidth statistics.
We will use their analysis to provide IPv6 progress reports.
We might also discover some bugs in tor's IPv6 stats code during the first analysis.
See proposal 313, section 8.2, metrics analysis part:
​https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n355
The definitions of these statistics are in proposal 313, sections 4 and 5:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n118https://gitlab.torproject.org/legacy/trac/-/issues/33263Prop 313: 4. Collect IPv6 Bandwidth Stats on Relays and Bridges2020-06-13T15:51:15ZteorProp 313: 4. Collect IPv6 Bandwidth Stats on Relays and BridgesWe propose that relays (and bridges) collect IPv6 consumed bandwidth
statistics.
To minimise development and testing effort, we propose re-using the existing
"bw_array" code in rephist.c.
(We might want to move this code into separate ...We propose that relays (and bridges) collect IPv6 consumed bandwidth
statistics.
To minimise development and testing effort, we propose re-using the existing
"bw_array" code in rephist.c.
(We might want to move this code into separate relay-only code and header files, because it is relay-specific.)
In particular, tor currently counts these bandwidth statistics:
* read,
* write,
* dir_read, and
* dir_write.
We propose adding the following bandwidth statistics:
* ipv6_read, and
* ipv6_write.
(The IPv4 statistics can be calculated by subtracting the IPv6 statistics
from the existing total consumed bandwidth statistics.)
We believe that collecting IPv6 consumed bandwidth statistics is about as
safe as the existing IPv4+IPv6 total consumed bandwidth statistics.
See proposal 313, section 4:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n118Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33260Add option to filter graphed OnionPerf results by relay fingerprint2020-06-13T18:04:09ZKarsten LoesingAdd option to filter graphed OnionPerf results by relay fingerprintWe want to provide the option to filter OnionPerf measurements by relay fingerprint to see how overall performance would change when removing relays with certain properties. The idea is that the user provides a set of fingerprints, eithe...We want to provide the option to filter OnionPerf measurements by relay fingerprint to see how overall performance would change when removing relays with certain properties. The idea is that the user provides a set of fingerprints, either as command-line arguments or contained in a flat file, and we exclude all measurements with those relays in the path. Ideally, we would make it possible for different sets of excluded relays to be plotted as different CDFs in the same graph.https://gitlab.torproject.org/legacy/trac/-/issues/33258Add CSV file export of graphed data2020-06-13T18:04:08ZKarsten LoesingAdd CSV file export of graphed dataWe're adding new graphs to OnionPerf, but we should expect developers wanting to play with the underlying data and feeding it into their own tools. We should provide an option to export plotted data to a CSV file, or maybe we should just...We're adding new graphs to OnionPerf, but we should expect developers wanting to play with the underlying data and feeding it into their own tools. We should provide an option to export plotted data to a CSV file, or maybe we should just write such a file whenever we write graph image files.Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/33257Add CDF-DL graph2020-06-13T18:04:06ZKarsten LoesingAdd CDF-DL graphWe're planning to add the CDF-DL graph discussed in #33076 to OnionPerf. This ticket is for adding that graph to the set of graphs produced by OnionPerf's visualization mode.We're planning to add the CDF-DL graph discussed in #33076 to OnionPerf. This ticket is for adding that graph to the set of graphs produced by OnionPerf's visualization mode.Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/33223Prop 311: 4.3.1. Don't Publish Descriptor if IPv6 ORPort is Unreachable2020-06-13T15:50:57ZteorProp 311: 4.3.1. Don't Publish Descriptor if IPv6 ORPort is UnreachableThis ticket depends on the protocol version ticket #33226.
Since it stops relays publishing their descriptors, it needs to be
tested on the public tor network, see #33229 and #33230.
If IPv6 reachability checks fail, relays (and bridge...This ticket depends on the protocol version ticket #33226.
Since it stops relays publishing their descriptors, it needs to be
tested on the public tor network, see #33229 and #33230.
If IPv6 reachability checks fail, relays (and bridges) should refuse to
publish their descriptors, if:
* enough existing relays support IPv6 extends, and
* the IPv6 address was explicitly configured by the operator
(rather than guessed using [Proposal 312: Relay Auto IPv6 Address]).
Directory authorities may perform reachability checks, and warn if those
checks fail. But they always publish their descriptors.
We set a threshold of consensus relays for reliable IPv6 ORPort checks:
* at least 30 relays, and
* at least 1% of the total consensus weight,
must support IPv6 extends.
We chose these parameters so that the number of relays is triple the
number of directory authorities, and the consensus weight is high enough
to support occasional reachability circuits.
In small networks with:
* less than 2000 relays, or
* a total consensus weight of zero,
the threshold should be the minimum tor network size to test reachability:
* at least 2 relays, excluding this relay.
(Note: we may increase this threshold to 3 or 4 relays if we discover a
higher minimum during testing.)
If the current consensus satisfies this threshold, testing relays (and
bridges, but not directory authorities) that fail IPv6 ORPort reachability
checks should refuse to publish their descriptors.
To ensure an accurate threshold, testing relays should exclude:
* the testing relay itself, and
* relays that they will not use in testing circuits,
from the:
* relay count, and
* the numerator of the threshold percentage.
Typically, relays will be excluded if they are in the testing relay's:
* family,
* IPv4 address /16 network,
* IPv6 address /32 network (a requirement as of Tor 0.4.0.1-alpha),
unless EnforceDistinctSubnets is 0.
As a useful side-effect, these different thresholds for each relay family
will reduce the likelihood of the network flapping around the threshold.
If flapping has an impact on the network health, directory authorities
should set the AssumeIPv6Reachable consensus parameter. (See the next
section.)
See proposal 311, section 4.3.1:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n351https://gitlab.torproject.org/legacy/trac/-/issues/33220Prop 311: 3. Allow Relay IPv6 Extends2020-06-13T15:50:53ZteorProp 311: 3. Allow Relay IPv6 ExtendsRelays may make a new connection over IPv6 when:
* they have an IPv6 ORPort,
* there is no existing authenticated connection to the requested relay, and
* the extend cell contains an IPv6 ORPort.
If these conditions are satisfied,...Relays may make a new connection over IPv6 when:
* they have an IPv6 ORPort,
* there is no existing authenticated connection to the requested relay, and
* the extend cell contains an IPv6 ORPort.
If these conditions are satisfied, and the extend cell also contains an
IPv4 ORPort, we propose that the relay choose between an IPv4 and an IPv6
connection at random.
If the extend cell does not contain an IPv4 ORPort, we propose that the
relay connects over IPv6. (Relays should support IPv6-only extend cells,
even though they are not used to test relay reachability in this proposal.)
A successful IPv6 connection also requires that:
* the requested relay has an IPv6 ORPort.
But extending relays must not check the consensus for other relays' IPv6
information. Consensuses may be out of date, particularly when relays are
doing reachability checks for new IPv6 ORPorts.
From proposal 311, section 3:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n112teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33185Create a pipeline for gettor statistics2020-06-21T18:06:05ZCecylia BocovichCreate a pipeline for gettor statisticsWe have some functionality in place for collecting statistics for GetTor (see https://dip.torproject.org/torproject/anti-censorship/gettor-project/gettor/issues/10), but there is some work to be done in collecting these stats at regular ...We have some functionality in place for collecting statistics for GetTor (see https://dip.torproject.org/torproject/anti-censorship/gettor-project/gettor/issues/10), but there is some work to be done in collecting these stats at regular intervals and exporting them to CollecTor (if we decide that's what we want).
This ticket is for regularly collecting and putting GetTor statistics into a format that's useful for us and others.https://gitlab.torproject.org/legacy/trac/-/issues/33182Automate process from reporting bad relays to resolving them as much as possible2020-06-13T17:54:14ZGeorg KoppenAutomate process from reporting bad relays to resolving them as much as possibleCurrently we still have some manual steps in the process of badexiting relays between incoming reports and actual pushs to the dirauth-conf repo. We should eliminate those as good as we can.Currently we still have some manual steps in the process of badexiting relays between incoming reports and actual pushs to the dirauth-conf repo. We should eliminate those as good as we can.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/legacy/trac/-/issues/33180Fix issues with bad relay scanners2020-06-13T17:54:13ZGeorg KoppenFix issues with bad relay scannersOur bad relay scanners have accumulated a bunch of issues. We should go over them and resolve them while taking a fresh look on duplicated functionality and making plans about restructuring functionality.Our bad relay scanners have accumulated a bunch of issues. We should go over them and resolve them while taking a fresh look on duplicated functionality and making plans about restructuring functionality.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/legacy/trac/-/issues/33065metrics dirbytes graph either underreports or leaves out dir auths?2020-06-13T18:15:31ZRoger Dingledinemetrics dirbytes graph either underreports or leaves out dir auths?https://metrics.torproject.org/dirbytes.html
shows that on average about 1gbit/s is spent serving directory information.
But the investigations in #33018 show that moria1 by itself is pushing about 135mbit/s, and I hear gabelmoo is doin...https://metrics.torproject.org/dirbytes.html
shows that on average about 1gbit/s is spent serving directory information.
But the investigations in #33018 show that moria1 by itself is pushing about 135mbit/s, and I hear gabelmoo is doing even more than that.
The text under the graph says "directory mirrors", so maybe we are leaving out dir auths in this graph? In that case, it would be cool to change it to one of those layered graphs so you can see how much of the total is dir auths and how much of the total are other relays.
Or maybe we're somehow not adding up the right things? Step zero is to spot-check the current data and the current graphs and become convinced that indeed we're presenting the right data.
Originally reported in
https://trac.torproject.org/projects/tor/ticket/33018#comment:3Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/33051O1.4 - Measure the number of Tor relays that support IPv6 reachability checks2020-06-13T15:50:20ZGabagaba@torproject.orgO1.4 - Measure the number of Tor relays that support IPv6 reachability checksSee Proposal 313: Relay IPv6 Statistics:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt
Tor's IPv6 reachability checks are described in:
Proposal 311: Tor Relay IPv6 Reachability:
https://gitweb.torpro...See Proposal 313: Relay IPv6 Statistics:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt
Tor's IPv6 reachability checks are described in:
Proposal 311: Tor Relay IPv6 Reachability:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt
The child tickets are in proposal section order, but they will probably be implemented in this order:
Dependencies:
* Prop 311: 5. Declare Support for Subprotocol Version "Relay=3" (O1.1, #33226)
Implementation:
* Write a Script that Counts IPv6 Relays in the Consensus
Metrics Review:
* Check IPv6 Relay Consensus Counts Script
Internal Testing:
* Test IPv6 Relay Consensus Counts using Chutney (O1.3, #33267)
Public Tor Network Testing:
* Test IPv6 Relay Consensus Counts on the Tor Network
Monitoring:
* Monitor IPv6 Relay Counts in the Consensushttps://gitlab.torproject.org/legacy/trac/-/issues/33036Revive Twitter responder2020-06-21T18:06:04ZCecylia BocovichRevive Twitter responderThe Twitter responder is not currently working for GetTor. There are some outstanding issues here we need to address.The Twitter responder is not currently working for GetTor. There are some outstanding issues here we need to address.https://gitlab.torproject.org/legacy/trac/-/issues/33016Make sure we're measuring the reachability of our gettor distribution methods2020-06-21T18:06:03ZCecylia BocovichMake sure we're measuring the reachability of our gettor distribution methodsRight now users can download gettor links from 4 different locations:
- GitLab repository
- Github release
- Internet Archive
- Google Drive
We should make sure we use existing means for checking reachability (e.g., OONI) for keeping tr...Right now users can download gettor links from 4 different locations:
- GitLab repository
- Github release
- Internet Archive
- Google Drive
We should make sure we use existing means for checking reachability (e.g., OONI) for keeping track of whether these locations are reachable. If there is a region where none are reachable, we need to figure out new distribution aveneues.https://gitlab.torproject.org/legacy/trac/-/issues/32912Clean up gettor scripts2020-06-21T18:06:00ZHiroClean up gettor scriptsSome of the scripts under gettor/scripts do not work properly or do something that some other script covers.
These should be cleaned up.Some of the scripts under gettor/scripts do not work properly or do something that some other script covers.
These should be cleaned up.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/32910trace: Add tracepoints and userspace tracer support2020-06-13T15:49:51ZDavid Gouletdgoulet@torproject.orgtrace: Add tracepoints and userspace tracer supportTor-dev email: https://lists.torproject.org/pipermail/tor-dev/2019-December/014111.html
These are what needs to be done but they sorta need to be together to make sense thus this one ticket:
a. Add a series of tracepoints in tor code b...Tor-dev email: https://lists.torproject.org/pipermail/tor-dev/2019-December/014111.html
These are what needs to be done but they sorta need to be together to make sense thus this one ticket:
a. Add a series of tracepoints in tor code base. I propose to start with circuit and cell level tracepoints for now.
b. Add USDT (User Statically-Defined Tracing) probes support which is used by SystemTap, DTrace and perf.
c. Add LTTng support which if enable also emits USDT.
d. Integrate all this to our build system.
About(d), the consensus among the network team is that it should NEVER be enabled in production and should be a configure switch.
I believe if we add on top a torrc option, it might not be that useful in the end considering the configure switch but mainly it will degrade performance since the check needs to be at runtime for every tracepoint.Tor: 0.4.4.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/32783Investigate clusterfuzz build failures2020-06-13T15:49:16ZNick MathewsonInvestigate clusterfuzz build failuresOur clusterfuzz setup has run into some build issues; I need to figure it out.Our clusterfuzz setup has run into some build issues; I need to figure it out.Tor: 0.4.3.x-finalNick MathewsonNick Mathewson