The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2020-06-27T13:18:30Zhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33793Avoid some race conditions when running chutney networks in series2020-06-27T13:18:30ZteorAvoid some race conditions when running chutney networks in seriesWhen chutney runs multiple networks, one after the others, sometimes older tor instances are left running.
To avoid these issues, chutney can:
* wait longer after terminating tor processes
* cleanup pid files, so we don't accidentally t...When chutney runs multiple networks, one after the others, sometimes older tor instances are left running.
To avoid these issues, chutney can:
* wait longer after terminating tor processes
* cleanup pid files, so we don't accidentally terminate unrelated processesSponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33615Wait for at least 60 seconds for 0.3.5 and earlier to bootstrap2020-06-27T13:18:31ZteorWait for at least 60 seconds for 0.3.5 and earlier to bootstrapThe basic-min network is still a bit unstable, but only for 0.3.5. (And it's much more unrealiable on macOS than Linux.)
It seems like 0.3.5 needs around 60 seconds to bootstrap and sync descriptors. Later versions have microdescriptor ...The basic-min network is still a bit unstable, but only for 0.3.5. (And it's much more unrealiable on macOS than Linux.)
It seems like 0.3.5 needs around 60 seconds to bootstrap and sync descriptors. Later versions have microdescriptor download fixes, and they can bootstrap and verify in just over 20 seconds on basic-min.
So let's make the minimum wait time 70 seconds (3 consensuses), if any nodes in the network are running 0.3.5 (or earlier).Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33583Stop setting AssumeReachable on chutney relays and bridges2020-07-23T20:36:05ZteorStop setting AssumeReachable on chutney relays and bridgesWe need to stop setting AssumeReachable on relays and bridges in chutney networks, so they do reachability self-tests.
We should continue to set AssumeReachable on authorities (including the bridge authority), because it disables author...We need to stop setting AssumeReachable on relays and bridges in chutney networks, so they do reachability self-tests.
We should continue to set AssumeReachable on authorities (including the bridge authority), because it disables authority to relay reachability tests. (These tests are still performed on a half-hourly schedule, even in chutney networks. And they are out of scope for Sponsor 55.)
After we implement this change, we should also be able to implement legacy/trac#33581.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33379Make chutney wait for all relays in the consensus before verifying2020-06-27T13:18:32ZteorMake chutney wait for all relays in the consensus before verifyingAs part of legacy/trac#33232, we want chutney to check that all relays are in the consensus, then verify.As part of legacy/trac#33232, we want chutney to check that all relays are in the consensus, then verify.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33376Update the networks in Chutney's CI to match Tor's new test-network*2020-06-27T13:18:33ZteorUpdate the networks in Chutney's CI to match Tor's new test-network*We need to update the list of chutney networks in chutney's CI, to match the networks we added to Tor's `make test-network` and `make test-network-all` in legacy/trac#33334.We need to update the list of chutney networks in chutney's CI, to match the networks we added to Tor's `make test-network` and `make test-network-all` in legacy/trac#33334.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33353Split chutney's diagnostics into a new script2020-06-27T13:18:33ZteorSplit chutney's diagnostics into a new scriptChutney's failure diagnostics are currently in the Travis CI config file.
But we want to use them in tor's CI. And maybe chutney users want to use them as well.Chutney's failure diagnostics are currently in the Travis CI config file.
But we want to use them in tor's CI. And maybe chutney users want to use them as well.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33302Run bridges+hs-v23 as Chutney's default network2020-06-27T13:18:33ZteorRun bridges+hs-v23 as Chutney's default networkNow that 0.2.9 is obsolete, we can upgrade chutney's defaults.
Users who want to run unsupported tor versions < 0.3.2 can choose a legacy flavour, like `basic-min` or `bridges+hs-v2`.Now that 0.2.9 is obsolete, we can upgrade chutney's defaults.
Users who want to run unsupported tor versions < 0.3.2 can choose a legacy flavour, like `basic-min` or `bridges+hs-v2`.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33230Prop 311: 6.1. Ask Relay Operators to Test IPv6 Reachability2020-08-07T13:16:23ZteorProp 311: 6.1. Ask Relay Operators to Test IPv6 ReachabilitySee legacy/trac#33229 for advice on testing stages.
Write an email to the tor-relays list, asking relay operators to help test the proposal 311 IPv6 ORPort Reachability and Extends changes.
Once these changes are merged, volunteer rela...See legacy/trac#33229 for advice on testing stages.
Write an email to the tor-relays list, asking relay operators to help test the proposal 311 IPv6 ORPort Reachability and Extends changes.
Once these changes are merged, volunteer relay and bridge operators will be able to test them by:
* compiling from source,
* running nightly builds, or
* running alpha releases.
See proposal 311, section 6.1, relay operator test part:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n671Sponsor 55: Improving the Tor network’s IPv6 supportNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33229Prop 311: 6.1. Test IPv6 ORPort Reachability on the Tor Network2020-07-28T15:02:26ZteorProp 311: 6.1. Test IPv6 ORPort Reachability on the Tor NetworkTest the IPv6 ORPort Reachability and Extends changes in proposal 311, on the public tor network, with a small number of relays and bridges.
We can do public tests after each major feature is merged. We can
also wait for multiple featur...Test the IPv6 ORPort Reachability and Extends changes in proposal 311, on the public tor network, with a small number of relays and bridges.
We can do public tests after each major feature is merged. We can
also wait for multiple features to merge, and test them all.
Here are some useful features that we can test:
legacy/trac#33222 / legacy/trac#33226 IPv6 ORPort Reachability Self-Tests / Relay=3 Protocol
legacy/trac#33223 Don't Publish Descriptor if IPv6 ORPort is Unreachable
See proposal 311, section 6.1, initial public test part:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n671Sponsor 55: Improving the Tor network’s IPv6 supportNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33050O1.3 - Integration test Tor relays over IPv6 using chutney2020-08-14T12:37:42ZGabagaba@torproject.orgO1.3 - Integration test Tor relays over IPv6 using chutneyTest the implementation of Sponsor 55, using chutney.
For details, see:
Proposal 311: Tor Relay IPv6 Reachability:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt
Proposal 312: Tor Relay Automa...Test the implementation of Sponsor 55, using chutney.
For details, see:
Proposal 311: Tor Relay IPv6 Reachability:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt
Proposal 312: Tor Relay Automatic IPv6 Address Discovery:
https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv6-addr.txt
Proposal 313: Tor Relay IPv6 Statistics:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt
See the implementation order in:
* Objective 1.1 - legacy/trac#33048
* Objective 1.2 - legacy/trac#33049
* Objective 1.4 - legacy/trac#33051
* Objective 1.5 - legacy/trac#33052
Children:
* [x] tor#33280 Add a "make test-network-ipv6" target, which runs IPv6 chutney networks
* [x] chutney#33793 Avoid some race conditions when running chutney networks in series
* [x] chutney#33675 Search microdescriptor files for relay ed25519 keys
* [x] chutney#33676 Stop waiting a set time for microdescriptors
* [x] tor#33300 Add a basic IPv6 test to "make test-network"
* [x] chutney#33428 Make chutney check for relay microdescriptors before verifying
* [x] chutney#33302 Run bridges+hs-v23 as Chutney's default network
* [x] tor#33303 Travis: Only run IPv6 chutney tests on macOS
* [x] chutney#33304 Chutney tries to convert empty pid string to integer
* [x] tor#4631 Idea to make consensus voting more resistant
* [x] tor#32792 Copy chutney CI diagnostics to Tor's chutney job
* [ ] chutney#33825 Make Environ handle "in" and "get()" like a dict
* [x] chutney#33957 Unexpected keyword argument 'bufsize' in subprocess.check_output()
* [x] tor#33194 Sort Travis jobs in speed order
* [x] tor#33195 Require IPv6 tests in Travis CI
* [ ] tor#33582 Make bridges wait until they have bootstrapped, before publishing their descriptor
* [x] chutney#33583 Stop setting AssumeReachable on chutney relays and bridges
* [x] tor#28208 Run bridges+hs-v23 for make test-network
* [x] chutney#33333 Add a mixed+hs-v23-ipv6 network to chutney
* [x] tor#33334 Add a mixed+hs-v23-ipv6 network to tor's test-network
* [x] chutney#33595 Stop waiting for unchecked directory info
* [x] chutney#33596 Fix or disable mixed+hs-v2 for Tor 0.3.5
* [ ] chutney#33598 chutney does not fail on some SOCKS errors
* [x] chutney#33353 Split chutney's diagnostics into a new script
* [x] chutney#33609 Check that onion services have successfully posted descriptors before verifying
* [x] chutney#33228 Prop 311: 6.1. Test IPv6 ORPort Reachability using Chutney
* [x] tor#33358 Update dir-spec for consensus voting improvements
* [ ] chutney#33231 Prop 311: 6.3. Test Legacy Relays Accept IPv6 Extends using Chutney
* [x] chutney#33232 Test IPv4 Reachability using Chutney
* [x] chutney#33615 Wait for at least 60 seconds for 0.3.5 and earlier to bootstrap
* [x] chutney#33376 Update the networks in Chutney's CI to match Tor's new test-network*
* [x] chutney#33250 Test IPv4 Address Detection using Chutney
* [x] chutney#33251 Prop 312: 5.1. Test Relay IPv6 Addresses Discovery using Chutney
* [x] chutney#33378 Require chutney node bootstrap before running verify
* [x] chutney#33379 Make chutney wait for all relays in the consensus before verifying
* [ ] chutney#34037 Make chutney check tor's logs for reachability self-test success
* [x] tor#33918 Stop truncating IPv6 addresses in channel logsSponsor 55: Improving the Tor network’s IPv6 supporthttps://gitlab.torproject.org/tpo/core/tor/-/issues/40138Give configure-time warnings on API mismatch with Openssl headers and libraries2020-10-07T13:00:58ZNick MathewsonGive configure-time warnings on API mismatch with Openssl headers and librariesIn #33437, @jsw ran into a usability problem that happens when our configure script finds the openssl headers in one place but finds the libraries in another. Instead of complaining, the script succeeds, but later on compilation fails.
...In #33437, @jsw ran into a usability problem that happens when our configure script finds the openssl headers in one place but finds the libraries in another. Instead of complaining, the script succeeds, but later on compilation fails.
I'd like to improve our openssl detection, but it's a longstanding problem to do that without breaking backward compatibility with existing configure users.
So as a usability measure, let's give a better warning when we find ourselves in this situation, and let's make it hard to overlook.Tor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40136Revise doc/state-contents.txt to be accurate2021-09-16T14:21:52ZNick MathewsonRevise doc/state-contents.txt to be accurateThe doc/state-contents.txt file is out of date, and missing a bunch of documentation.
We should have accurate documentation for all of the state file's contents.
The following elements are undocumented:
* BWHistoryIPv6ReadEnds
* BWHi...The doc/state-contents.txt file is out of date, and missing a bunch of documentation.
We should have accurate documentation for all of the state file's contents.
The following elements are undocumented:
* BWHistoryIPv6ReadEnds
* BWHistoryIPv6ReadInterval
* BWHistoryIPv6ReadValues
* BWHistoryIPv6ReadMaxima
* BWHistoryIPv6WriteEnds
* BWHistoryIPv6WriteInterval
* BWHistoryIPv6WriteValues
* BWHistoryIPv6WriteMaxima
* Guard
* TotalBuildTimes
* CircuitBuildAbandonedCount
* "CircuitBuildTimeBin"
* "BuildtimeHistogram"
* "MinutesSinceUserActivity"
* "Dormant"
The follow elements are obsolete:
* "EntryGuard"
* "EntryGuardDownSince"
* "EntryGuardUnlistedSince"
* "EntryGuardAddedBy"
These are obsolete _and_ undocumented:
* "EntryGuardPathBias"
* "EntryGuardPathUseBias"
* HidServRevCounterTor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40135Debug log sr_state_get_start_time_of_current_protocol_run() appears ten of th...2020-09-22T17:49:23ZDavid Gouletdgoulet@torproject.orgDebug log sr_state_get_start_time_of_current_protocol_run() appears ten of thousands of time when starting torHere is an example of the log line:
> Sep 21 20:56:04.000 [debug] sr_state_get_start_time_of_current_protocol_run(): Current SRV proto run: Start of current round: 1600711200. Time elapsed: 64800 (3600)
For a very simple debug log of w...Here is an example of the log line:
> Sep 21 20:56:04.000 [debug] sr_state_get_start_time_of_current_protocol_run(): Current SRV proto run: Start of current round: 1600711200. Time elapsed: 64800 (3600)
For a very simple debug log of what it is suppose to be few thousands lines, that line is logged 10 times that so debug.log are enormous but in reality they should be much smaller.
We should ratelimit that line or just don't print it maybe.Tor: 0.4.5.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40126One last underflow in rend_cache tests2020-09-18T13:46:14ZNick MathewsonOne last underflow in rend_cache testsWe found two instances of this already in our tests: #40099 and #40125. In response, I tried running every one of our tests on its own, to see if there are other cases. It looks like we have one last failure:
```
rend_cache/free_all: S...We found two instances of this already in our tests: #40099 and #40125. In response, I tried running every one of our tests on its own, to see if there are other cases. It looks like we have one last failure:
```
rend_cache/free_all: Sep 17 14:00:06.862 [warn] rend_cache_decrement_allocation(): Bug:
Underflow in rend_cache_decrement_allocation (on Tor 0.4.5.0-alpha-dev 404c224c711714f0)
```Tor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40125hs-v2: The rend_cache/entry_free test underflows rend_cache_decrement_allocat...2021-06-23T17:22:08ZDavid Gouletdgoulet@torproject.orghs-v2: The rend_cache/entry_free test underflows rend_cache_decrement_allocation()For some reasons, the test `rend_cache/entry_free` recently started to fail with:
> rend_cache/entry_free: Sep 17 08:40:13.845 [warn] rend_cache_decrement_allocation(): Bug: Underflow in rend_cache_decrement_allocation (on Tor 0.4.5.0-a...For some reasons, the test `rend_cache/entry_free` recently started to fail with:
> rend_cache/entry_free: Sep 17 08:40:13.845 [warn] rend_cache_decrement_allocation(): Bug: Underflow in rend_cache_decrement_allocation (on Tor 0.4.5.0-alpha-dev 7eef9ced61e72b1d)
And this is actually quite obvious by looking at the test that it would underflow all the time since the total cache size is never incremented but we decrement it when freeing.
This requires backport for the CI to pass on all our versions.Tor: 0.4.5.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40124Incorrect key ID type used in some ed25519 certificates2022-07-07T00:48:31ZNick MathewsonIncorrect key ID type used in some ed25519 certificatesIn `cert-spec.txt` we specify several possible values for the CERT_KEY_TYPE field, in section A.4. But we don't actually use those: everywhere that we call `tor_cert_sign_impl()` , signed_key type is set to `SIGNED_KEY_TYPE_ED25519`.
...In `cert-spec.txt` we specify several possible values for the CERT_KEY_TYPE field, in section A.4. But we don't actually use those: everywhere that we call `tor_cert_sign_impl()` , signed_key type is set to `SIGNED_KEY_TYPE_ED25519`.
We should adjust the spec to clarify that current tor implementations behave, and (assuming it won't introduce compatibility issue) adjust Tor relay behavior to conform to the spec. We should probably leave onion service behavior alone.Tor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40103Coverity issues in test_glob()2020-09-17T13:20:47ZNick MathewsonCoverity issues in test_glob()Coverity Scan has reported some issues in `test_glob()`. These are test-only, so they don't reflect a security vulnerability, but we should fix them soon to keep our coverity backlog small.Coverity Scan has reported some issues in `test_glob()`. These are test-only, so they don't reflect a security vulnerability, but we should fix them soon to keep our coverity backlog small.Tor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40102Make options_init_from_torrc smaller2021-09-16T14:21:52ZJigsaw52Make options_init_from_torrc smalleroptions_init_from_torrc is a very large function that implements some of the tor command line options inside it. These could be easily split into each own function.options_init_from_torrc is a very large function that implements some of the tor command line options inside it. These could be easily split into each own function.Tor: 0.4.5.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40076Assertion buf->tail failed in buf_assert_ok at src/lib/buf/buffers.c:9192021-02-05T21:03:36ZDimitris ApostolouAssertion buf->tail failed in buf_assert_ok at src/lib/buf/buffers.c:919Latest master on macOS 10.15.6
```
Jul 30 09:50:12.000 [notice] Bootstrapped 100% (done): Done
Jul 30 09:50:13.000 [err] tor_assertion_failed_: Bug: src/lib/buf/buffers.c:919: buf_assert_ok: Assertion buf->tail failed; aborting. (on Tor...Latest master on macOS 10.15.6
```
Jul 30 09:50:12.000 [notice] Bootstrapped 100% (done): Done
Jul 30 09:50:13.000 [err] tor_assertion_failed_: Bug: src/lib/buf/buffers.c:919: buf_assert_ok: Assertion buf->tail failed; aborting. (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: Tor 0.4.5.0-alpha-dev (git-9164d7c75e619182): Assertion buf->tail failed in buf_assert_ok at src/lib/buf/buffers.c:919: . Stack trace: (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 0 tor 0x000000010c69dbd9 log_backtrace_impl + 89 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 1 tor 0x000000010c699a92 tor_assertion_failed_ + 322 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 2 tor 0x000000010c675d6f buf_assert_ok + 495 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 3 tor 0x000000010c4e3d7c assert_connection_ok + 220 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 4 tor 0x000000010c4eecc6 conn_read_callback + 326 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 5 libevent-2.1.7.dylib 0x000000010ca0bfde event_process_active_single_queue + 1186 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 6 libevent-2.1.7.dylib 0x000000010ca08ddb event_base_loop + 1174 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 7 tor 0x000000010c4f1115 do_main_loop + 309 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 8 tor 0x000000010c623be8 tor_run_main + 296 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 9 tor 0x000000010c542f1d tor_main + 125 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 10 tor 0x000000010c4de01b main + 27 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
Jul 30 09:50:13.000 [err] Bug: 11 libdyld.dylib 0x00007fff67dd5cc9 start + 1 (on Tor 0.4.5.0-alpha-dev 9164d7c75e619182)
```Tor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40075CFG_PORT_AUTO reported as 128451502020-07-30T16:50:32ZNick MathewsonCFG_PORT_AUTO reported as 12845150`Configuration port ORPort 12845150 superseded by ORPort [::1]:12845150`
This is new in 0.4.5.`Configuration port ORPort 12845150 superseded by ORPort [::1]:12845150`
This is new in 0.4.5.Tor: 0.4.5.x-freezeNick MathewsonNick Mathewson