Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T14:38:30Zhttps://gitlab.torproject.org/legacy/trac/-/issues/13112Some things are probably broken when we advertise multiple ORPorts and only s...2020-06-13T14:38:30ZAndrea ShepardSome things are probably broken when we advertise multiple ORPorts and only some are reachableObservations on reachability testing made while fixing #12160:
- We only have a 1-bit notion of reachability; if we get an incoming non-local connection, we assume reachability in onionskin_answer() and call router_orport_found_reachab...Observations on reachability testing made while fixing #12160:
- We only have a 1-bit notion of reachability; if we get an incoming non-local connection, we assume reachability in onionskin_answer() and call router_orport_found_reachable() to publish a descriptor.
- We should have a reachability bit per *advertised* ORPort to determine its inclusion in the published descriptor, and publish if and only if we have one or more reachable ORPorts.
- To implement this, we need a way to link incoming testing circuits to a particular advertised ORPort; we don't know this from the port the underlying channel was listening on because reverse proxies might make this not one-to-one in general.
- Arma suggests in IRC that netinfo cells know the IP the connection was attempted on and if they were extended with a port number they might provide a sufficient mechanism.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/20647Tor and Chutney CI improvements2020-06-13T15:03:19ZChelsea KomloTor and Chutney CI improvementsWe currently run chutney tests on every Tor and Chutney branch and pull request. We deal with non-determinism by allowing a small number of chutney failures, before reporting an overall test failure.
But sometimes failures are hard to d...We currently run chutney tests on every Tor and Chutney branch and pull request. We deal with non-determinism by allowing a small number of chutney failures, before reporting an overall test failure.
But sometimes failures are hard to diagnose. This ticket is a parent ticket for improvements that help us diagnose chutney failures.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/25153Specify how PrivCount in Tor statistics are configured and interpreted2020-06-13T15:21:41ZteorSpecify how PrivCount in Tor statistics are configured and interpretedIn ~~prop#280~~ prop#288, we specified the counters, noise, and secure aggregation for PrivCount in Tor.
Now we need to specify:
* how long each collection round goes for
* how we determine which counters are collected
* how we configur...In ~~prop#280~~ prop#288, we specified the counters, noise, and secure aggregation for PrivCount in Tor.
Now we need to specify:
* how long each collection round goes for
* how we determine which counters are collected
* how we configure the amount of noise for each counter
* how Metrics can interpret the final results
We should also try to answer the unanswered questions in prop280.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26944Privcount blinding and encryption: Add tests2020-06-13T15:28:43ZteorPrivcount blinding and encryption: Add testsAdd tests as recommended in https://trac.torproject.org/projects/tor/ticket/25669#comment:15Add tests as recommended in https://trac.torproject.org/projects/tor/ticket/25669#comment:15Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26945Privcount blinding and encryption: always enable i1282020-06-13T15:28:44ZteorPrivcount blinding and encryption: always enable i128See https://trac.torproject.org/projects/tor/ticket/25669#comment:12See https://trac.torproject.org/projects/tor/ticket/25669#comment:12Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26957Privcount blinding and encryption: Derive copy and debug where possible2020-06-13T15:28:48ZteorPrivcount blinding and encryption: Derive copy and debug where possibleThe privcount shamir structs are missing copy and debug implementations.
komlo: are these useful?
If they are, we should derive them, and then enable the missing_copy_implementations and missing_debug_implementations warnings.The privcount shamir structs are missing copy and debug implementations.
komlo: are these useful?
If they are, we should derive them, and then enable the missing_copy_implementations and missing_debug_implementations warnings.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26958Privcount blinding and encryption: run clippy on travis rust nightly2020-06-13T15:28:48ZteorPrivcount blinding and encryption: run clippy on travis rust nightlyWe'll need to fix or disable a lot of warnings for clippy.We'll need to fix or disable a lot of warnings for clippy.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26970Privcount: plan the modules and components2020-06-13T15:28:49ZteorPrivcount: plan the modules and componentsReplying to [26953#comment:3 chelseakomlo]:
> Is the idea that this project will remain external to core tor, or will this one day be merged into the core codebase? Definitely having CI in the short term seems wise either way.
That's a ...Replying to [26953#comment:3 chelseakomlo]:
> Is the idea that this project will remain external to core tor, or will this one day be merged into the core codebase? Definitely having CI in the short term seems wise either way.
That's a good question, nickm and I haven't discussed it yet. And I think we'd benefit from your advice.
For PrivCount in Tor, we need to produce the following components:
* a Rust "Data Collector" module in Tor that does blinding, encryption, and noise, based on a config
* a separate "Tally Reporter" binary that does unblinding, decryption, aggregation, and reporting, based on a config
* some tools for creating and validating configurations
One possible design is:
* Rust modules for blinding/encryption, noise, aggregation, reporting, and config
* Glue code and module imports for the Tor Data Collector
* Application code and module imports for the Tally Reporter
* Application code and module imports for the tools
Split off https://trac.torproject.org/projects/tor/ticket/26953?replyto=3#comment:3Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27146Mismatched digest in 0.3.3.9 and master mixed chutney network2020-06-13T15:32:51ZteorMismatched digest in 0.3.3.9 and master mixed chutney networkWhen I run master i386 and Tor 0.3.3.9 x68_64 in a mixed chutney network, I see the following error:
```
Detail: chutney/tools/warnings.sh /Users/USER/tor/chutney/tools/../net/nodes.1534263100
Warning: Unable to add signatures to consens...When I run master i386 and Tor 0.3.3.9 x68_64 in a mixed chutney network, I see the following error:
```
Detail: chutney/tools/warnings.sh /Users/USER/tor/chutney/tools/../net/nodes.1534263100
Warning: Unable to add signatures to consensus: Mismatched digest. Number: 102
Exit 255
```
master x86_64 and Tor 0.3.3.9 x68_64 in a mixed chutney network also get the same error:
```
Detail: chutney/tools/warnings.sh /Users/USER/tor/chutney/tools/../net/nodes.1534285790
Warning: Unable to add signatures to consensus: Mismatched digest. Number: 102
Exit 255
```
We should fix this issue if it affects Linux and BSD. If it doesn't, maybe we should downgrade our support for authorities on macOS.
If we want to diagnose this issue, implementing #20625 and #4539 would help.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27161Add make check-rustfmt to make check2020-06-13T15:29:25ZteorAdd make check-rustfmt to make checkIn #26972, we discovered that stable and nightly rustfmt disagree about the formatting of some of Tor's Rust code. Beta currently follows nightly.
On 13 September, the current beta branch will become stable, and the formatting differenc...In #26972, we discovered that stable and nightly rustfmt disagree about the formatting of some of Tor's Rust code. Beta currently follows nightly.
On 13 September, the current beta branch will become stable, and the formatting differences should go away. (Rust releases happen every 6 weeks, and the last one was on 2 August 2018: https://github.com/rust-lang/rust/blob/master/RELEASES.md .)
Here's what we need to do:
* cherry-pick the final commit on the branch 'rustfmt-travis' from https://gitgud.io/onionk/tor.git or https://github.com/teor2345/tor.git
* run rustfmt on any Rust code added since #26972, and commit the results
* make sure "make check" passes with Rust stable, beta, and nightly
* merge the changesTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27162Travis: consider running CI on beta, nightly, and tor's lowest supported rust2020-06-13T15:29:26ZteorTravis: consider running CI on beta, nightly, and tor's lowest supported rustAt the moment, Tor's Travis CI runs on stable rust, and our Appveyor (Windows) CI doesn't run rust at all (#26954).
But in our privcount_shamir work (#25669), we've discovered some important bugs by running beta and nightly.
Since Rust...At the moment, Tor's Travis CI runs on stable rust, and our Appveyor (Windows) CI doesn't run rust at all (#26954).
But in our privcount_shamir work (#25669), we've discovered some important bugs by running beta and nightly.
Since Rust releases every 6 weeks, we should run CI on rust stable and beta, so that we catch any show-stoppers before they are stable.
We might also want to consider an allow_failures nightly build.
If we choose to support a lower version of rust than stable (when a major distro freezes on a lower version), we should also CI that version.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27284Check IPv6 exit policies on microdescs2020-06-13T15:30:01ZteorCheck IPv6 exit policies on microdescsIn node_exit_policy_rejects_all(), we check IPv4 and IPv6 policies on ri, but on md we only check IPv4:
```
else if (node->md)
return node->md->exit_policy == NULL ||
short_policy_is_reject_star(node->md->exit_policy);
```
O...In node_exit_policy_rejects_all(), we check IPv4 and IPv6 policies on ri, but on md we only check IPv4:
```
else if (node->md)
return node->md->exit_policy == NULL ||
short_policy_is_reject_star(node->md->exit_policy);
```
One way to fix this issue is to refactor the existing code to check a new policy_is_reject_star, and then populate policy_is_reject_star when the md is parsed. (Like we already do with the ri.)Tor: 0.4.2.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/27381Bad consensus diffs on 0.3.4 and later [with chutney]2020-06-13T15:30:28ZteorBad consensus diffs on 0.3.4 and later [with chutney]An 0.3.4 authority triggered this error on another authority:
```
Summary: nodes
Detail: chutney/tools/warnings.sh /Users/base/chutney/net/nodes.1535545138
Warning: Could not apply consensus diff received from server '127.0.0.1:7001' Num...An 0.3.4 authority triggered this error on another authority:
```
Summary: nodes
Detail: chutney/tools/warnings.sh /Users/base/chutney/net/nodes.1535545138
Warning: Could not apply consensus diff received from server '127.0.0.1:7001' Number: 1
Warning: Expected: D086F055796593B54352BA551F2CFC59A7CB9CE1C0E0BDC339C7B16BEB233D41; found: C81D53C1D18C0889500FB727B40C69E4716D7E9C8F3D8731D0F43B2F44BC03B1 Number: 1
Warning: Refusing to apply consensus diff because the base consensus doesn't match the digest as found in the consensus diff header. Number: 1
```
But the authority that delivered the bad consensus diff seems fine:
```
Aug 29 22:19:16.813 [notice] Time to vote.
Aug 29 22:19:16.817 [notice] Choosing valid-after time in vote as 2018-08-29 12:19:20: consensus_set=1, last_interval=10
Aug 29 22:19:16.834 [notice] Vote posted.
Aug 29 22:19:16.847 [notice] Uploaded a vote to dirserver 127.0.0.1:7002
Aug 29 22:19:16.855 [notice] Uploaded a vote to dirserver 127.0.0.1:7003
Aug 29 22:19:16.857 [notice] Uploaded a vote to dirserver 127.0.0.1:7000
Aug 29 22:19:17.818 [notice] Time to fetch any votes that we're missing.
Aug 29 22:19:18.700 [notice] Got a signature from 127.0.0.1. Queuing it for the next consensus.
Aug 29 22:19:18.822 [notice] Time to compute a consensus.
Aug 29 22:19:18.826 [notice] Computed bandwidth weights for Case 2b1 (Wgg=weight_scale, Wmd=Wgd) with v10: G=1 M=1 E=1 D=1 T=4
Aug 29 22:19:18.829 [notice] Bandwidth-weight Case 1 is verified and valid.
Aug 29 22:19:18.835 [notice] Computed bandwidth weights for Case 2b1 (Wgg=weight_scale, Wmd=Wgd) with v10: G=1 M=1 E=1 D=1 T=4
Aug 29 22:19:18.840 [notice] Bandwidth-weight Case 1 is verified and valid.
Aug 29 22:19:18.843 [notice] Added a signature for test000a from pending.
Aug 29 22:19:18.850 [notice] Added 2 pending signatures while building consensus.
Aug 29 22:19:18.850 [notice] Consensus computed; uploading signature(s)
Aug 29 22:19:18.855 [notice] Signature(s) posted.
Aug 29 22:19:18.865 [notice] Uploaded signature(s) to dirserver 127.0.0.1:7002
Aug 29 22:19:18.867 [notice] Uploaded signature(s) to dirserver 127.0.0.1:7003
Aug 29 22:19:18.880 [notice] Uploaded signature(s) to dirserver 127.0.0.1:7000
Aug 29 22:19:19.177 [notice] Got a signature from 127.0.0.1. Adding it to the pending consensus.
Aug 29 22:19:19.179 [notice] Added a signature for test003aOLD from 127.0.0.1.
Aug 29 22:19:19.228 [notice] Got a signature from 127.0.0.1. Adding it to the pending consensus.
Aug 29 22:19:19.229 [notice] Added a signature for test002aOLD from 127.0.0.1.
Aug 29 22:19:19.828 [notice] Time to fetch any signatures that we're missing.
Aug 29 22:19:20.829 [notice] Time to publish the consensus and discard old votes
Aug 29 22:19:20.835 [notice] Published ns consensus
Aug 29 22:19:20.854 [notice] Published microdesc consensus
Aug 29 22:19:26.839 [notice] Time to vote.
```
As does the authority that received the bad diff, until it gets the diff:
```
Aug 29 22:19:17.146 [notice] Time to vote.
Aug 29 22:19:17.150 [notice] Choosing valid-after time in vote as 2018-08-29 12:19:20: consensus_set=1, last_interval=10
Aug 29 22:19:17.160 [notice] Vote posted.
Aug 29 22:19:17.174 [notice] Uploaded a vote to dirserver 127.0.0.1:7000
Aug 29 22:19:17.175 [notice] Uploaded a vote to dirserver 127.0.0.1:7001
Aug 29 22:19:17.178 [notice] Uploaded a vote to dirserver 127.0.0.1:7002
Aug 29 22:19:18.146 [notice] Time to fetch any votes that we're missing.
Aug 29 22:19:18.703 [notice] Got a signature from 127.0.0.1. Queuing it for the next consensus.
Aug 29 22:19:18.861 [notice] Got a signature from 127.0.0.1. Queuing it for the next consensus.
Aug 29 22:19:19.146 [notice] Time to compute a consensus.
Aug 29 22:19:19.149 [notice] Computed bandwidth weights for Case 2b1 (Wgg=weight_scale, Wmd=Wgd) with v10: G=1 M=1 E=1 D=1 T=4
Aug 29 22:19:19.151 [notice] Bandwidth-weight Case 1 is verified and valid.
Aug 29 22:19:19.154 [notice] Computed bandwidth weights for Case 2b1 (Wgg=weight_scale, Wmd=Wgd) with v10: G=1 M=1 E=1 D=1 T=4
Aug 29 22:19:19.156 [notice] Bandwidth-weight Case 1 is verified and valid.
Aug 29 22:19:19.158 [notice] Added a signature for test000a from pending.
Aug 29 22:19:19.162 [notice] Added a signature for test001a from pending.
Aug 29 22:19:19.166 [notice] Added 4 pending signatures while building consensus.
Aug 29 22:19:19.167 [notice] Consensus computed; uploading signature(s)
Aug 29 22:19:19.170 [notice] Signature(s) posted.
Aug 29 22:19:19.181 [notice] Uploaded signature(s) to dirserver 127.0.0.1:7002
Aug 29 22:19:19.194 [notice] Uploaded signature(s) to dirserver 127.0.0.1:7001
Aug 29 22:19:19.198 [notice] Uploaded signature(s) to dirserver 127.0.0.1:7000
Aug 29 22:19:19.231 [notice] Got a signature from 127.0.0.1. Adding it to the pending consensus.
Aug 29 22:19:19.232 [notice] Added a signature for test002aOLD from 127.0.0.1.
Aug 29 22:19:20.146 [notice] Time to fetch any signatures that we're missing.
Aug 29 22:19:21.148 [notice] Time to publish the consensus and discard old votes
Aug 29 22:19:21.152 [notice] Published ns consensus
Aug 29 22:19:21.158 [notice] Published microdesc consensus
Aug 29 22:19:21.180 [warn] Refusing to apply consensus diff because the base consensus doesn't match the digest as found in the consensus diff header.
Aug 29 22:19:21.181 [warn] Expected: D086F055796593B54352BA551F2CFC59A7CB9CE1C0E0BDC339C7B16BEB233D41; found: C81D53C1D18C0889500FB727B40C69E4716D7E9C8F3D8731D0F43B2F44BC03B1
Aug 29 22:19:21.181 [warn] Could not apply consensus diff received from server '127.0.0.1:7001'
Aug 29 22:19:27.147 [notice] Time to vote.
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27382Bad valid-after time in 0.3.3 and 0.3.42020-06-13T15:30:29ZteorBad valid-after time in 0.3.3 and 0.3.40.3.4 authorities will post votes that other authorities on 0.3.3 or 0.3.4 consider too old:
```
$ grep -r "21:05:33" . | grep -v -e info -e debug | sort -k 3
./000a/notice.log:Aug 29 21:05:33.786 [notice] Time to vote.
./000a/notice.log...0.3.4 authorities will post votes that other authorities on 0.3.3 or 0.3.4 consider too old:
```
$ grep -r "21:05:33" . | grep -v -e info -e debug | sort -k 3
./000a/notice.log:Aug 29 21:05:33.786 [notice] Time to vote.
./000a/notice.log:Aug 29 21:05:33.790 [notice] Choosing valid-after time in vote as 2018-08-29 11:05:35: consensus_set=0, last_interval=5
./000a/notice.log:Aug 29 21:05:33.803 [notice] Vote posted.
./002aOLD/notice.log:Aug 29 21:05:33.811 [warn] Rejected vote from 127.0.0.1 ("Bad valid-after time").
./002aOLD/notice.log:Aug 29 21:05:33.811 [warn] Rejecting vote from 127.0.0.1 with valid-after time of 2018-08-29 11:05:35; we were expecting 2018-08-29 11:05:40
./003aOLD/notice.log:Aug 29 21:05:33.812 [warn] Rejected vote from 127.0.0.1 ("Bad valid-after time").
./003aOLD/notice.log:Aug 29 21:05:33.812 [warn] Rejecting vote from 127.0.0.1 with valid-after time of 2018-08-29 11:05:35; we were expecting 2018-08-29 11:05:40
./001a/notice.log:Aug 29 21:05:33.813 [warn] Rejected vote from 127.0.0.1 ("Bad valid-after time").
./001a/notice.log:Aug 29 21:05:33.813 [warn] Rejecting vote from 127.0.0.1 with valid-after time of 2018-08-29 11:05:35; we were expecting 2018-08-29 11:05:40
./000a/notice.log:Aug 29 21:05:33.815 [warn] http status 400 ("Bad valid-after time") response after uploading vote to dirserver '127.0.0.1:7002'. Please correct.
./000a/notice.log:Aug 29 21:05:33.818 [warn] http status 400 ("Bad valid-after time") response after uploading vote to dirserver '127.0.0.1:7003'. Please correct.
./000a/notice.log:Aug 29 21:05:33.820 [warn] http status 400 ("Bad valid-after time") response after uploading vote to dirserver '127.0.0.1:7001'. Please correct.
./000a/notice.log:Aug 29 21:05:33.835 [notice] Bootstrapped 80%: Connecting to the Tor network
```
But I've also seen 0.3.3 authorities reject their own votes for this reason.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27468CI: add builds with the latest clang and gcc2020-06-13T15:30:48ZteorCI: add builds with the latest clang and gccWe could add the latest gcc and clang to our Travis matrix.
Travis is running older gcc and clang versions:
* Linux: Ubuntu's older clang and gcc
* macOS: Apple's custom clang, no gcc
Appveyor is already running gcc 8.2, and it seems t...We could add the latest gcc and clang to our Travis matrix.
Travis is running older gcc and clang versions:
* Linux: Ubuntu's older clang and gcc
* macOS: Apple's custom clang, no gcc
Appveyor is already running gcc 8.2, and it seems to be updated every 6-12 months.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27778Travis: add link-time optimisation to detect function signature mismatches2020-06-13T15:31:44ZteorTravis: add link-time optimisation to detect function signature mismatchesFor #27772, nickm used:
```
./configure CC=clang CFLAGS='-O2 -flto' LDFLAGS='-flto' RANLIB=/bin/true AR=llvm-ar NM=llvm-nm
```For #27772, nickm used:
```
./configure CC=clang CFLAGS='-O2 -flto' LDFLAGS='-flto' RANLIB=/bin/true AR=llvm-ar NM=llvm-nm
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27906PrivCount noise specification2020-06-13T15:32:19ZteorPrivCount noise specificationLet's write a spec for PrivCount noise:
* how noise is determined for each statistic
* how much noise each relay adds
* how to interpret the final result
* how to avoid obvious pitfallsLet's write a spec for PrivCount noise:
* how noise is determined for each statistic
* how much noise each relay adds
* how to interpret the final result
* how to avoid obvious pitfallsTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28037Document how to fix or disable failing CI2020-06-13T15:32:51ZteorDocument how to fix or disable failing CIWe want our CI to always be passing.
Sometimes, we can fix the bug quickly, or revert the commit.
But if the CI environment is broken, we need to disable the failing test cases until the bug is fixed.
In doc/HACKING, document how to di...We want our CI to always be passing.
Sometimes, we can fix the bug quickly, or revert the commit.
But if the CI environment is broken, we need to disable the failing test cases until the bug is fixed.
In doc/HACKING, document how to disable failing CI jobs.
In the Travis and Appveyor configs, summarise or give an example of how to disable a failing CI config.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28228In Chutney's debug mode, dump all tor warning logs to stderr as soon as they ...2020-06-13T13:30:40ZteorIn Chutney's debug mode, dump all tor warning logs to stderr as soon as they appearCurrently we wait until the end, which isn't ideal.Currently we wait until the end, which isn't ideal.https://gitlab.torproject.org/legacy/trac/-/issues/284523. implement rounding gap smoothing as in proposal 2762020-06-13T16:14:24Zteor3. implement rounding gap smoothing as in proposal 276I have a branch for this, but it doesn't work yet.I have a branch for this, but it doesn't work yet.sbws: 1.2.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/28582Document the load-balancing goal for sbws2020-06-13T15:34:30ZteorDocument the load-balancing goal for sbwsWe should include a section early in the bandwidth spec that documents the load balancing goal for sbws.
Here's my initial draft:
```
Bandwidth scanners should have a well-defined network equilibrium goal.
For sbws and Torflow without ...We should include a section early in the bandwidth spec that documents the load balancing goal for sbws.
Here's my initial draft:
```
Bandwidth scanners should have a well-defined network equilibrium goal.
For sbws and Torflow without PID control, the network equilibrium goals are:
1. clients experience reasonably consistent performance
2. clients experience performance that is as fast as possible, without compromising goal 1
These performance goals include both throughput and latency.
For Torflow with PID control, the goal was:
1. Each relay has exactly the same spare capacity for an additional stream
This goal was unachievable because Tor did not provide enough feedback on circuit failure.
These non-goals are common suggestions:
1. Bandwidth is allocated equally across relays
2. All relay bandwidth is used
These goals are unachievable, because they conflict with the consistent client performance goal.
```
Based on mike's response from this thread:
```
I believe quite strongly that even if the Tor network gets faster on
average, if this comes at the cost of increased performance variance,
user experience and perceived speed of Tor will be much worse. There's
nothing more annoying than a system that is *usually* fast enough to do
what you need it to do, but fails to be fast enough for that activity at
unpredictable times.
```
https://lists.torproject.org/pipermail/tor-dev/2018-August/013419.htmlTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29006PrivCount proof of concept: add noise to counters2020-06-13T15:36:21ZteorPrivCount proof of concept: add noise to countersWe need to add noise to PrivCount counters to protect user activity. We split this noise between the Data Collectors (DCs), so that the final aggregate count includes enough noise to protect at least one user's activity.
For consumed ba...We need to add noise to PrivCount counters to protect user activity. We split this noise between the Data Collectors (DCs), so that the final aggregate count includes enough noise to protect at least one user's activity.
For consumed bandwidth, we can calculate an average user's activity by dividing the total consumed bandwidth by the total number of users. Then, we split this noise across all the DCs (the relays that support version 1 of the PrivCount protocol), using each relay's consensus weight fraction.
DCs need to add noise when we start, and when each consensus arrives, based on our own consensus weight fraction in that consensus. (How do we deal with the off-by-one error here? Weight by the time between the last consensus, the round end/start, and the time we'll try to fetch the next consensus?)
We'll need to add excess noise to compensate for relay failures, and malicious relays.
We should just use the easiest Gaussian sampling method available. Adding any noise is an improvement for almost all of our statistics - we can deal with floating point issues later.
We should keep track of:
* ConsensusCount - the number of consensuses we've seen
* PrivCountConsensusWeightFraction - the consensus weight of this relay, divided by the consensus weight of all relays supporting PrivCount
For each counter, we should keep track of:
* NoiseVarianceAmount - the total variance (standard deviation squared) of all noise added to this counter. We use variance because it's additive. (And standard deviation is not.)Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29007PrivCount proof of concept: implement debugging for PrivCount2020-06-13T15:36:21ZteorPrivCount proof of concept: implement debugging for PrivCountLet's debug PrivCount by splitting statistics into multiple categories:
* public statistics: information that is already public
* private statistics: information that is sensitive
We can safely log public statistics at any log level. We...Let's debug PrivCount by splitting statistics into multiple categories:
* public statistics: information that is already public
* private statistics: information that is sensitive
We can safely log public statistics at any log level. We can publish public statistics' unencrypted, per-relay values and noise amounts in extrainfo descriptors.
We need a debug mode to log private statistics above info level. When we are in debug mode, and not using any public authorities, we can publish private statistics' unencrypted, per-relay values and noise amounts in extrainfo descriptors.
We'll work out other debugging mechanisms during the proof of concept process.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29008PrivCount proof of concept: add a PrivCount module, torrc option, and protocol2020-06-13T15:36:22ZteorPrivCount proof of concept: add a PrivCount module, torrc option, and protocolWe should make PrivCount into an optional module, with a compile-time option. If Rust is off, or the PrivCount option is off, PrivCount will be disabled.
When PrivCount is enabled, it will advertise support for the PrivCount protocol ve...We should make PrivCount into an optional module, with a compile-time option. If Rust is off, or the PrivCount option is off, PrivCount will be disabled.
When PrivCount is enabled, it will advertise support for the PrivCount protocol versions and statistics versions it supports.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29010PrivCount proof of concept: work out how to do CI for PrivCount using chutney2020-06-13T15:36:23ZteorPrivCount proof of concept: work out how to do CI for PrivCount using chutneyWe can do CI for PrivCount using chutney:
* launch a standard chutney network
* launch the Tally Reporter (TR)
* run some data through the network
* check the results using a chutney script
If we want to check extrainfo descriptors agai...We can do CI for PrivCount using chutney:
* launch a standard chutney network
* launch the Tally Reporter (TR)
* run some data through the network
* check the results using a chutney script
If we want to check extrainfo descriptors against PrivCount statistics, we'll need to publish them more often in test networks.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29027PrivCount proof of concept: put the PrivCount statistics in a stats/ file2020-06-13T15:36:31ZteorPrivCount proof of concept: put the PrivCount statistics in a stats/ fileMaybe we should put all the stats in files while we're at it.Maybe we should put all the stats in files while we're at it.Tor: unspecified