Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:51:43Zhttps://gitlab.torproject.org/legacy/trac/-/issues/33407Make chutney bridge authorities publish bridges in their networkstatus-bridges2020-06-13T15:51:43ZteorMake chutney bridge authorities publish bridges in their networkstatus-bridgesThis issue depends on the tor bridge descriptor upload fix in #33582, or robust reachability self-tests in #33222.
Chutney bridge authorities don't have any bridges in their networkstatus-bridges. That's a problem, because we want to ch...This issue depends on the tor bridge descriptor upload fix in #33582, or robust reachability self-tests in #33222.
Chutney bridge authorities don't have any bridges in their networkstatus-bridges. That's a problem, because we want to check networkstatus-bridges for the reachability checks in #33232.
Once bridges upload their descriptors, we can make chutney check the bridge authority's networkstatus-bridges for bridge nicknames (this ticket).
We can only do the networkstatus-bridges check on tor versions with the #33222 or #33582 fixes. So we'll need to check for:
* the next tor 0.4.4-alpha version, or
* an environmental variable that enables these tests.
We don't have to do these fixes, because it should be enough to test relay reachability. But we would risk breaking bridge reachability tests, and not knowing about it until after a release.Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33353Split chutney's diagnostics into a new script2020-06-13T13:31:32ZteorSplit chutney's diagnostics into a new scriptChutney's failure diagnostics are currently in the Travis CI config file.
But we want to use them in tor's CI. And maybe chutney users want to use them as well.Chutney's failure diagnostics are currently in the Travis CI config file.
But we want to use them in tor's CI. And maybe chutney users want to use them as well.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33303Travis: Only run IPv6 chutney tests on macOS2020-06-13T15:51:24ZteorTravis: Only run IPv6 chutney tests on macOSTo speed up the macOS chutney job, we only want to run IPv6 tests.To speed up the macOS chutney job, we only want to run IPv6 tests.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33302Run bridges+hs-v23 as Chutney's default network2020-06-13T13:31:30ZteorRun bridges+hs-v23 as Chutney's default networkNow that 0.2.9 is obsolete, we can upgrade chutney's defaults.
Users who want to run unsupported tor versions < 0.3.2 can choose a legacy flavour, like `basic-min` or `bridges+hs-v2`.Now that 0.2.9 is obsolete, we can upgrade chutney's defaults.
Users who want to run unsupported tor versions < 0.3.2 can choose a legacy flavour, like `basic-min` or `bridges+hs-v2`.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33300Add a basic IPv6 test to "make test-network"2020-06-13T15:51:23ZteorAdd a basic IPv6 test to "make test-network"We need to test IPv6 during Sponsor 55.
(If it's available on the local machine.)We need to test IPv6 during Sponsor 55.
(If it's available on the local machine.)Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33280Add a "make test-network-ipv6" target, which runs IPv6 chutney networks2020-06-13T15:51:19ZteorAdd a "make test-network-ipv6" target, which runs IPv6 chutney networksPart of #33195.Part of #33195.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33195Require IPv6 tests in Travis CI2020-06-13T15:50:47ZteorRequire IPv6 tests in Travis CIWhile we're actively changing our IPv6 code, we should make sure that the macOS chutney IPv6 tests pass in Travis CI.
While I'm doing this change, I will see if there are:
* some redundant jobs I can delete, or
* some inactive areas th...While we're actively changing our IPv6 code, we should make sure that the macOS chutney IPv6 tests pass in Travis CI.
While I'm doing this change, I will see if there are:
* some redundant jobs I can delete, or
* some inactive areas that I can make fast_finish.
I think our Rust build might be a good candidate for fast_finish, we haven't changed that code much in about a year. But I should check with the team before making this change.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33194Sort Travis jobs in speed order2020-06-13T15:50:46ZteorSort Travis jobs in speed orderAnd remove a redundant job.
We try to put the slowest Travis jobs first, so CI finishes as early as possible.
But we put the optional macOS chutney / IPv6 job last, because it's very very slow, so we don't wait for it to finish.
This ...And remove a redundant job.
We try to put the slowest Travis jobs first, so CI finishes as early as possible.
But we put the optional macOS chutney / IPv6 job last, because it's very very slow, so we don't wait for it to finish.
This ticket is required for Sponsor 55, because we need to do it to make IPv6 tests mandatory.Tor: 0.4.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/32792Copy chutney CI diagnostics to Tor's chutney job2020-06-13T15:49:19ZteorCopy chutney CI diagnostics to Tor's chutney jobIn #32630, we improved chutney's CI diagnostics on failure. We should also run those diagnostics in tor's chutney CI job.In #32630, we improved chutney's CI diagnostics on failure. We should also run those diagnostics in tor's chutney CI job.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/32721Allow chutney users to disable tor's sandbox at runtime2020-06-13T13:31:14ZteorAllow chutney users to disable tor's sandbox at runtimeIn #32240, we discovered that tor's sandbox doesn't work on recent Ubuntu versions. So we need to disable the sandbox on those CI jobs.
One good way to implement this change, is by making it a first-class chutney feature.In #32240, we discovered that tor's sandbox doesn't work on recent Ubuntu versions. So we need to disable the sandbox on those CI jobs.
One good way to implement this change, is by making it a first-class chutney feature.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/32240Tor Travis: Make chutney work on Xenial and Bionic images2020-06-13T15:47:22ZteorTor Travis: Make chutney work on Xenial and Bionic imagesWe get weird permissions errors like this:
```
FAIL: basic-min
Detail: chutney/tools/warnings.sh /home/travis/build/teor2345/tor/chutney/net/nodes.1571845015
Warning: Can't create/check datadirectory /home/travis/build/teor2345/tor/chutn...We get weird permissions errors like this:
```
FAIL: basic-min
Detail: chutney/tools/warnings.sh /home/travis/build/teor2345/tor/chutney/net/nodes.1571845015
Warning: Can't create/check datadirectory /home/travis/build/teor2345/tor/chutney/net/nodes/002r Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/000a/key-pinning-journal" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/001a/key-pinning-journal" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/cached-certs": Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/cached-consensus" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/cached-descriptors" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/cached-extrainfo" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/cached-microdesc-consensus" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/cached-microdescs" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/cached-microdescs.new": Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/unverified-consensus" for mmap(): Permission denied Number: 1
Warning: Could not open "/home/travis/build/teor2345/tor/chutney/net/nodes/003c/unverified-microdesc-consensus" for mmap(): Permission denied Number: 1
Warning: Directory /home/travis/build/teor2345/tor/chutney/net/nodes/002r cannot be read: Permission denied Number: 1
Warning: Error initializing keys; exiting Number: 1
Warning: Error loading key-pinning journal: Permission denied Number: 2
```Tor: 0.2.9.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/30455Improve documentation for chutney warnings in "make test-network-all"2020-06-13T15:41:26ZNick MathewsonImprove documentation for chutney warnings in "make test-network-all"It appears that in fb32c522320430f, we added a second call to test-network.sh inside our test-network-all loop. Now the code looks like this:
```
for f in $$flavors; do \
$(SHELL) $(top_srcdir)/test-driver --test-name $$f --log-file...It appears that in fb32c522320430f, we added a second call to test-network.sh inside our test-network-all loop. Now the code looks like this:
```
for f in $$flavors; do \
$(SHELL) $(top_srcdir)/test-driver --test-name $$f --log-file $(TEST_NETWORK_ALL_LOG_DIR)/$$f.log --trs-file $(TEST_NETWORK_ALL_LOG_DIR)/$$f.trs $(TEST_NETWORK_ALL_DRIVER_FLAGS) $(top_srcdir)/src/test/test-network.sh --flavor $$f $(TEST_NETWORK_FLAGS); \
$(top_srcdir)/src/test/test-network.sh $(TEST_NETWORK_WARNING_FLAGS); \
done; \
```
I might be wrong, but it looks to me like we're calling test-network.sh twice in each loop: once through `test-driver`, and once directly.
I'm not going to work on this till teor is back, though, since there are dragons here that I do not understand.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28220Create a chutney network with a single authority, and make sure it bootstraps2020-06-13T15:33:24ZteorCreate a chutney network with a single authority, and make sure it bootstrapsDepends on #28203, which logs bootstrap progress.
It looks like there are a lot of race conditions in chutney and tor. We can eliminate most of these race conditions by launching a single authority, and seeing if it bootstraps reliably....Depends on #28203, which logs bootstrap progress.
It looks like there are a lot of race conditions in chutney and tor. We can eliminate most of these race conditions by launching a single authority, and seeing if it bootstraps reliably. Then we can narrow down the source of chutney's instability.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28204Make chutney log the verify progress for each circuit2020-06-13T13:30:39ZteorMake chutney log the verify progress for each circuitChutney lists the circuits that it's building, but it doesn't say which ones have succeeded, and which ones have failed.
Let's log (some of) the following events for each circuit:
* circuit open / connect or fail to connect
* first data...Chutney lists the circuits that it's building, but it doesn't say which ones have succeeded, and which ones have failed.
Let's log (some of) the following events for each circuit:
* circuit open / connect or fail to connect
* first data sent or fail to send
* first data received or fail to verify
* each 10%? of data, or each second, or something?
* last data sent
* last data receivedhttps://gitlab.torproject.org/legacy/trac/-/issues/28203Make chutney log the bootstrap progress for each node2020-07-28T23:20:05ZteorMake chutney log the bootstrap progress for each nodeTo solve #20473 and implement #22132, we need to know when nodes have bootstrapped.
Bootstrap progress is also useful when debugging failures.To solve #20473 and implement #22132, we need to know when nodes have bootstrapped.
Bootstrap progress is also useful when debugging failures.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28013run test-network-all in Travis CI2020-06-13T15:32:46ZTaylor Yurun test-network-all in Travis CIWe should run `make test-network-all` in Travis CI for the tor repository. #27912 tracks the corresponding work for the chutney repository.We should run `make test-network-all` in Travis CI for the tor repository. #27912 tracks the corresponding work for the chutney repository.Tor: 0.4.1.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/legacy/trac/-/issues/27295make test-network-all should clear the test_network_log directory before running2020-06-13T15:30:04Zteormake test-network-all should clear the test_network_log directory before runningIf it doesn't, stale results in the directory can cause the build to fail.If it doesn't, stale results in the directory can cause the build to fail.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/26164Create HS network in chutney with client-auth enabled2020-06-13T13:30:28ZGeorge KadianakisCreate HS network in chutney with client-auth enabledTo improve our HS integration tests as part of `make test-network-all` we should make a network template similar to `hs` etc. where the HS is client-auth-protected, and the client needs to use the client auth token to submit data.
This ...To improve our HS integration tests as part of `make test-network-all` we should make a network template similar to `hs` etc. where the HS is client-auth-protected, and the client needs to use the client auth token to submit data.
This might be a bit of a hurdle to setup, since we would need to first start up the HS, so that it generates the client auth tokens, and then we would need to hax the client torrc to make it happen.https://gitlab.torproject.org/legacy/trac/-/issues/25692onion_extend_cpath: Non-fatal assertion info || client failed.2020-06-13T15:26:13ZNick Mathewsononion_extend_cpath: Non-fatal assertion info || client failed.When running "make test-network-all", I see a bunch of these errors:
```
Apr 02 18:46:57.375 [warn] Bug: Non-fatal assertion info || client failed in onion_extend_cpath at src/or/circuitbuild.c:2772. Stack trace: (on Tor 0.3.4.0-alpha-d...When running "make test-network-all", I see a bunch of these errors:
```
Apr 02 18:46:57.375 [warn] Bug: Non-fatal assertion info || client failed in onion_extend_cpath at src/or/circuitbuild.c:2772. Stack trace: (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(log_backtrace+0x43) [0x55a0c255ee03] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(tor_bug_occurred_+0xb9) [0x55a0c257a3c9] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(circuit_establish_circuit+0x732) [0x55a0c24b2ae2] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(circuit_launch_by_extend_info+0x8c) [0x55a0c24c487c] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(circuit_build_needed_circs+0x2e7) [0x55a0c24c4ef7] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(+0x8e078) [0x55a0c2430078] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /lib64/libevent-2.0.so.5(event_base_loop+0x7a9) [0x7f2fa8e2df19] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(do_main_loop+0x245) [0x55a0c2430bf5] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(tor_run_main+0x1015) [0x55a0c24330b5] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(tor_main+0x3a) [0x55a0c242b79a] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(main+0x19) [0x55a0c242b529] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /lib64/libc.so.6(__libc_start_main+0xea) [0x7f2fa7468f2a] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Bug: /home/nickm/src/tor/src/or/tor(_start+0x2a) [0x55a0c242b57a] (on Tor 0.3.4.0-alpha-dev 386116d52ac5f869)
Apr 02 18:46:57.375 [warn] Failed to find node for hop #1 of our path. Discarding this circuit.
```
I don't seem to get this on 0.3.3, and neither does Teor.Tor: 0.3.3.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/22430Add next gen HS support to chutney2020-06-13T15:09:42ZGeorge KadianakisAdd next gen HS support to chutneyWe need a networks file and a torrc template for hs-v3 in chutney.We need a networks file and a torrc template for hs-v3 in chutney.teorteor