Chutney issueshttps://gitlab.torproject.org/tpo/core/chutney/-/issues2020-07-23T20:54:13Zhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30208Add missing features to Chutney's CI and test script2020-07-23T20:54:13ZteorAdd missing features to Chutney's CI and test scriptThere are some chutney features that don't have working tests.
We should create working tests, fix bugs, and add them to CI (like legacy/trac#29729) and the test script (legacy/trac#30066).There are some chutney features that don't have working tests.
We should create working tests, fix bugs, and add them to CI (like legacy/trac#29729) and the test script (legacy/trac#30066).https://gitlab.torproject.org/tpo/core/chutney/-/issues/30210Add a sleep after failure in test-network --forgiving2020-07-23T20:51:56ZteorAdd a sleep after failure in test-network --forgivingWhen we use test-network --forgiving in chutney's CI, sometimes every network run fails. Maybe this happens because of load on the machine, or some other factor.
I wonder if adding a sleep after failure will help.When we use test-network --forgiving in chutney's CI, sometimes every network run fails. Maybe this happens because of load on the machine, or some other factor.
I wonder if adding a sleep after failure will help.https://gitlab.torproject.org/tpo/core/chutney/-/issues/30211Work out why bridges+hs-v2 is unstable in CI2020-07-23T20:52:16ZteorWork out why bridges+hs-v2 is unstable in CIhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30279Test IPv6-only v3 onion services in Chutney's CI, once homebrew tor stable su...2020-06-27T13:18:38ZteorTest IPv6-only v3 onion services in Chutney's CI, once homebrew tor stable supports themWhen legacy/trac#23588 merges, we should replace single-onion-ipv6-md with single-onion-v23-ipv6-md in Chutney's CI.When legacy/trac#23588 merges, we should replace single-onion-ipv6-md with single-onion-v23-ipv6-md in Chutney's CI.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30459Let chutney tell Tor whether a network is supported2020-06-27T13:18:38ZNick MathewsonLet chutney tell Tor whether a network is supportedWe should do this anyway to simplify Tor's test-networks-all logic, but it will be helpful in order to have PT tests that can check whether the right PTs are installed.We should do this anyway to simplify Tor's test-networks-all logic, but it will be helpful in order to have PT tests that can check whether the right PTs are installed.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30820Show the correct macOS tor versions in chutney's CI2020-06-27T13:18:37ZteorShow the correct macOS tor versions in chutney's CII left out some environmental variables in legacy/trac#29729.I left out some environmental variables in legacy/trac#29729.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30826Chutney travis: remove 0.3.4, add 0.4.12020-06-27T13:18:37ZNick MathewsonChutney travis: remove 0.3.4, add 0.4.1The 0.4.1 branch is now separate; the 0.3.4 branches are now deprecated. We should update Chutney's travis configuration accordingly.The 0.4.1 branch is now separate; the 0.3.4 branches are now deprecated. We should update Chutney's travis configuration accordingly.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30876Remove 0.3.4 and experimental-0.4.0 from chutney's CI, and note that stable i...2020-06-27T13:18:37ZteorRemove 0.3.4 and experimental-0.4.0 from chutney's CI, and note that stable is now 0.4.0We removed some builds as part of legacy/trac#30836, so we need to fix CI.We removed some builds as part of legacy/trac#30836, so we need to fix CI.Tor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30928macOS Travis Homebrew cache has expired, failing some builds2020-06-27T13:18:37ZteormacOS Travis Homebrew cache has expired, failing some buildsThe Homebrew cache in the Travis macOS image has expired (or desynced), failing our chutney and stem builds:
```
Tapped 0 formulae (172 files, 244.5KB)
Error: Your Homebrew is outdated. Please run `brew update`.
Error: Kernel.exit
```
ht...The Homebrew cache in the Travis macOS image has expired (or desynced), failing our chutney and stem builds:
```
Tapped 0 formulae (172 files, 244.5KB)
Error: Your Homebrew is outdated. Please run `brew update`.
Error: Kernel.exit
```
https://travis-ci.org/torproject/stem/jobs/547493246#L87
https://travis-ci.org/torproject/chutney/jobs/548092876#L95
Our tor builds don't fail, but they aren't running shellcheck on macOS any more. That's probably ok.
```
Tapped 0 formulae (172 files, 244.5KB)
Error: Your Homebrew is outdated. Please run `brew update`.
Error: Kernel.exit
```
https://travis-ci.org/torproject/tor/jobs/547856188#L104
There should be a shellcheck here:
```
sfcgal 1.3.5
sqlite 3.24.0
```
https://travis-ci.org/torproject/tor/jobs/547856188#L182
There should be a result here:
```
if command -v shellcheck; then \
find "." -name "*.sh" -not -path "./src/ext/*" -exec shellcheck {} +; \
if [ -d "./scripts/test" ]; then \
shellcheck ./scripts/test/cov-diff ./scripts/test/coverage; \
fi; \
if [ -e "./contrib/dirauth-tools/nagios-check-tor-authority-cert" ]; then \
shellcheck "./contrib/dirauth-tools/nagios-check-tor-authority-cert"; \
fi; \
if [ -e "./contrib/client-tools/torify" ]; then \
shellcheck "./contrib/client-tools/torify"; \
fi; \
if [ -d "./scripts/git" ]; then \
shellcheck ./scripts/git/*.git-hook; \
fi; \
fi
```
https://travis-ci.org/torproject/tor/jobs/547856188#L3149
We can fix this in a few different ways:
1. make shellcheck optional in the chutney and stem Travis CI
* it will run on Linux, but not macOS. That's ok.
2. add the `update: true` key to our Travis macOS builds
* https://docs.travis-ci.com/user/installing-dependencies/#installing-packages-on-macos
* a full update makes macOS builds slower, so we should open a ticket to eventually remove it
3. allow_failure on the chutney and stem macOS Travis jobs
* https://docs.travis-ci.com/user/build-matrix/#rows-that-are-allowed-to-failAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32630Chutney Travis: Make chutney work on Bionic images2020-06-27T13:18:36ZteorChutney Travis: Make chutney work on Bionic imagesIf we fix legacy/trac#32240 in Tor, we should do the same thing for Chutney's Travis config.If we fix legacy/trac#32240 in Tor, we should do the same thing for Chutney's Travis config.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32631Chutney Travis: Use the latest dependencies2020-06-27T13:18:36ZteorChutney Travis: Use the latest dependenciesUse the latest dependencies in Chutney Travis:
* Linux and macOS images
* tor versions
* python versionsUse the latest dependencies in Chutney Travis:
* Linux and macOS images
* tor versions
* python versionsteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32721Allow chutney users to disable tor's sandbox at runtime2020-07-14T22:24:30ZteorAllow chutney users to disable tor's sandbox at runtimeIn legacy/trac#32240, we discovered that tor's sandbox doesn't work on recent Ubuntu versions. So we need to disable the sandbox on those CI jobs.
One good way to implement this change, is by making it a first-class chutney feature.In legacy/trac#32240, we discovered that tor's sandbox doesn't work on recent Ubuntu versions. So we need to disable the sandbox on those CI jobs.
One good way to implement this change, is by making it a first-class chutney feature.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32731Add __future__ imports to every chutney python file2020-06-27T13:18:36ZteorAdd __future__ imports to every chutney python fileTo make extra sure any changes to our python 2.7 code are compatible with python 3, we should add `__future__` imports to every chutney python file.To make extra sure any changes to our python 2.7 code are compatible with python 3, we should add `__future__` imports to every chutney python file.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32757Travis: Update the pypy builds for chutney2020-06-27T13:18:36ZteorTravis: Update the pypy builds for chutneyNow we know how to get pypy 2 and 3 in xenial on Travis, we should do it for chutney.Now we know how to get pypy 2 and 3 in xenial on Travis, we should do it for chutney.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32758Travis: Update the macOS image for chutney2020-06-27T13:18:36ZteorTravis: Update the macOS image for chutneyteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32775Remove 0.2.9 from chutney's CI2020-06-27T13:18:36ZteorRemove 0.2.9 from chutney's CIOn 1 January 2020, we won't be supporting 0.2.9 any more.On 1 January 2020, we won't be supporting 0.2.9 any more.https://gitlab.torproject.org/tpo/core/chutney/-/issues/32820Move the chutney pypy jobs to 0.4.2 xenial2020-06-27T13:18:36ZteorMove the chutney pypy jobs to 0.4.2 xenialTor master will soon be unsupported on Travis xenial, because Travis xenial only has OpenSSL 1.1.0. (See legacy/trac#31820.)
So we need to move the pypy jobs to Tor 0.4.2 xenial, because pypy isn't packaged for Travis Bionic yet.Tor master will soon be unsupported on Travis xenial, because Travis xenial only has OpenSSL 1.1.0. (See legacy/trac#31820.)
So we need to move the pypy jobs to Tor 0.4.2 xenial, because pypy isn't packaged for Travis Bionic yet.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/32969Chutney 'bootstrap-network.sh' shouldn't overwrite 'CHUTNEY_DATA_DIR'2020-06-27T13:18:35ZoparaChutney 'bootstrap-network.sh' shouldn't overwrite 'CHUTNEY_DATA_DIR'When you set up a tor network with something like:
```
tools/test-network.sh --flavor basic --net-dir /tmp/chutney-net
```
the `tools/test-network.sh` script will set `$CHUTNEY_DATA_DIR` based on the value of `--net-dir` and then call ...When you set up a tor network with something like:
```
tools/test-network.sh --flavor basic --net-dir /tmp/chutney-net
```
the `tools/test-network.sh` script will set `$CHUTNEY_DATA_DIR` based on the value of `--net-dir` and then call `tools/bootstrap-network.sh`. This bootstrapping script will then overwrite the variable `$CHUTNEY_DATA_DIR` if that directory doesn't exist or if the path is relative. This causes unexpected behavior when the user explicitly sets the `--net-dir` option. For example, Chutney works fine if you provide it with a directory that doesn't exist (it will create that directory automatically), so there is no need for the `tools/bootstrap-network.sh` script to unexpectedly change it to `$CHUTNEY_PATH/net`. The `tools/bootstrap-network.sh` also does not need to change it to an absolute path since Chutney [does this automatically](https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L144).
My suggestion is to remove the following lines from `tools/bootstrap-network.sh`:
```
# Get a working net path
if [ ! -d "$CHUTNEY_DATA_DIR" ]; then
# looks like a broken path: use the chutney path as a base
export CHUTNEY_DATA_DIR="$CHUTNEY_PATH/net"
fi
if [ -d "$PWD/$CHUTNEY_DATA_DIR" ]; then
# looks like a relative path: make chutney path absolute
export CHUTNEY_DATA_DIR="$PWD/$CHUTNEY_DATA_DIR"
fi
```https://gitlab.torproject.org/tpo/core/chutney/-/issues/32972Chutney doesn't remove temporary files after processing warnings2020-06-27T13:18:35ZoparaChutney doesn't remove temporary files after processing warningsWhen the Chutney test-network script ends, it displays warnings from the tor logs. It summarizes/filters them in the `show_warnings()` function in `tools/warnings.sh`. This function creates two files with `LOGS=$(mktemp)` and `FILTERED_L...When the Chutney test-network script ends, it displays warnings from the tor logs. It summarizes/filters them in the `show_warnings()` function in `tools/warnings.sh`. This function creates two files with `LOGS=$(mktemp)` and `FILTERED_LOGS=$(mktemp)` but never deletes them when it's done.
The proposal is to add `rm "${LOGS}" "${FILTERED_LOGS}"` at the end of the function.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33023Chutney 'bootstrap-network.sh' starts a different default network flavor than...2020-06-27T13:18:35ZoparaChutney 'bootstrap-network.sh' starts a different default network flavor than listed in the usage commentsSmall documentation problem in Chutney's `bootstrap-network.sh`.
The usage comment says the following:
```
# Usage:
# tools/bootstrap-network.sh [network-flavour]
# network-flavour: one of the files in the networks directory,
# ...Small documentation problem in Chutney's `bootstrap-network.sh`.
The usage comment says the following:
```
# Usage:
# tools/bootstrap-network.sh [network-flavour]
# network-flavour: one of the files in the networks directory,
# (default: 'basic')
```
but the actual code says the following:
```
# Set the variables for the chutney network flavour
export NETWORK_FLAVOUR=${NETWORK_FLAVOUR:-"bridges+hs-v2"}
[ -n "$1" ] && { NETWORK_FLAVOUR=$1; shift; }
export CHUTNEY_NETWORK="$CHUTNEY_PATH/networks/$NETWORK_FLAVOUR"
```
As you can see, the comments say that the default network is 'basic', but the code actually starts 'bridges+hs-v2'. Reference: https://github.com/torproject/chutney/blob/5a9271104fec6c5b9bb23d2574596dc9d7c0a2c4/tools/bootstrap-network.shhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33041Chutney error message says to use 'TOR_DIR' instead of 'CHUTNEY_TOR'2020-06-27T13:18:35ZoparaChutney error message says to use 'TOR_DIR' instead of 'CHUTNEY_TOR'When you attempt to run Chutney using a command like:
```
./chutney configure networks/basic
```
it will print the following error message if it cannot find `tor` or `tor-gencert`:
```
Cannot find the tor-gencert binary at 'tor-gencer...When you attempt to run Chutney using a command like:
```
./chutney configure networks/basic
```
it will print the following error message if it cannot find `tor` or `tor-gencert`:
```
Cannot find the tor-gencert binary at 'tor-gencert' for the command
line 'tor-gencert --create-identity-key --passphrase-fd 0 <snip>'.
Set the TOR_DIR environment variable to the directory containing
tor-gencert.
```
Chutney (the Python code) does not actually use this `TOR_DIR` environment variable (it's only used by `tools/test-network.sh` and related scripts). Instead the error message should say that the user should set the `CHUTNEY_TOR` and `CHUTNEY_TOR_GENCERT` environment variables.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33050O1.3 - Integration test Tor relays over IPv6 using chutney2020-08-14T12:37:42ZGabagaba@torproject.orgO1.3 - Integration test Tor relays over IPv6 using chutneyTest the implementation of Sponsor 55, using chutney.
For details, see:
Proposal 311: Tor Relay IPv6 Reachability:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt
Proposal 312: Tor Relay Automa...Test the implementation of Sponsor 55, using chutney.
For details, see:
Proposal 311: Tor Relay IPv6 Reachability:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt
Proposal 312: Tor Relay Automatic IPv6 Address Discovery:
https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv6-addr.txt
Proposal 313: Tor Relay IPv6 Statistics:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt
See the implementation order in:
* Objective 1.1 - legacy/trac#33048
* Objective 1.2 - legacy/trac#33049
* Objective 1.4 - legacy/trac#33051
* Objective 1.5 - legacy/trac#33052
Children:
* [x] tor#33280 Add a "make test-network-ipv6" target, which runs IPv6 chutney networks
* [x] chutney#33793 Avoid some race conditions when running chutney networks in series
* [x] chutney#33675 Search microdescriptor files for relay ed25519 keys
* [x] chutney#33676 Stop waiting a set time for microdescriptors
* [x] tor#33300 Add a basic IPv6 test to "make test-network"
* [x] chutney#33428 Make chutney check for relay microdescriptors before verifying
* [x] chutney#33302 Run bridges+hs-v23 as Chutney's default network
* [x] tor#33303 Travis: Only run IPv6 chutney tests on macOS
* [x] chutney#33304 Chutney tries to convert empty pid string to integer
* [x] tor#4631 Idea to make consensus voting more resistant
* [x] tor#32792 Copy chutney CI diagnostics to Tor's chutney job
* [ ] chutney#33825 Make Environ handle "in" and "get()" like a dict
* [x] chutney#33957 Unexpected keyword argument 'bufsize' in subprocess.check_output()
* [x] tor#33194 Sort Travis jobs in speed order
* [x] tor#33195 Require IPv6 tests in Travis CI
* [ ] tor#33582 Make bridges wait until they have bootstrapped, before publishing their descriptor
* [x] chutney#33583 Stop setting AssumeReachable on chutney relays and bridges
* [x] tor#28208 Run bridges+hs-v23 for make test-network
* [x] chutney#33333 Add a mixed+hs-v23-ipv6 network to chutney
* [x] tor#33334 Add a mixed+hs-v23-ipv6 network to tor's test-network
* [x] chutney#33595 Stop waiting for unchecked directory info
* [x] chutney#33596 Fix or disable mixed+hs-v2 for Tor 0.3.5
* [ ] chutney#33598 chutney does not fail on some SOCKS errors
* [x] chutney#33353 Split chutney's diagnostics into a new script
* [x] chutney#33609 Check that onion services have successfully posted descriptors before verifying
* [x] chutney#33228 Prop 311: 6.1. Test IPv6 ORPort Reachability using Chutney
* [x] tor#33358 Update dir-spec for consensus voting improvements
* [ ] chutney#33231 Prop 311: 6.3. Test Legacy Relays Accept IPv6 Extends using Chutney
* [x] chutney#33232 Test IPv4 Reachability using Chutney
* [x] chutney#33615 Wait for at least 60 seconds for 0.3.5 and earlier to bootstrap
* [x] chutney#33376 Update the networks in Chutney's CI to match Tor's new test-network*
* [x] chutney#33250 Test IPv4 Address Detection using Chutney
* [x] chutney#33251 Prop 312: 5.1. Test Relay IPv6 Addresses Discovery using Chutney
* [x] chutney#33378 Require chutney node bootstrap before running verify
* [x] chutney#33379 Make chutney wait for all relays in the consensus before verifying
* [ ] chutney#34037 Make chutney check tor's logs for reachability self-test success
* [x] tor#33918 Stop truncating IPv6 addresses in channel logsSponsor 55: Improving the Tor network’s IPv6 supporthttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33163Update list of Tor versions in Chutney's Travis CI2020-06-27T13:18:35ZteorUpdate list of Tor versions in Chutney's Travis CI* 0.2.9 and 0.4.0 are unsupported
* 0.4.2 is stable* 0.2.9 and 0.4.0 are unsupported
* 0.4.2 is stableteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33228Prop 311: 6.1. Test IPv6 ORPort Reachability using Chutney2020-06-27T13:18:35ZteorProp 311: 6.1. Test IPv6 ORPort Reachability using ChutneyTest the IPv6 ORPort Reachability and Extends changes in proposal 311, using chutney networks with AssumeReachable disabled.
(Chutney currently enables AssumeReachable by default.)
See proposal 311, section 6.1:
https://gitweb.torproje...Test the IPv6 ORPort Reachability and Extends changes in proposal 311, using chutney networks with AssumeReachable disabled.
(Chutney currently enables AssumeReachable by default.)
See proposal 311, section 6.1:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n671
Chutney already implements some of these tests, but we still need a fix for a bridge descriptor upload bug, see tor legacy/trac#33582 and chutney legacy/trac#33407.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33232Test IPv4 Reachability using Chutney2020-06-27T13:18:34ZteorTest IPv4 Reachability using ChutneyBefore we start making changes to tor's IPv6 code in legacy/trac#33048, we need to get IPv4 reachability tests working in chutney.Before we start making changes to tor's IPv6 code in legacy/trac#33048, we need to get IPv4 reachability tests working in chutney.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33250Test IPv4 Address Detection using Chutney2020-06-27T13:18:34ZteorTest IPv4 Address Detection using ChutneyBefore we start making changes to tor's IPv6 code in legacy/trac#33049, we need to get IPv4 address detection tests working in chutney.
Since chutney runs on private addresses, these tests might be difficult to write, or they might not ...Before we start making changes to tor's IPv6 code in legacy/trac#33049, we need to get IPv4 address detection tests working in chutney.
Since chutney runs on private addresses, these tests might be difficult to write, or they might not tell us much about how tor relays behave on the public tor network. So we shouldn't spend much time on them.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33251Prop 312: 5.1. Test Relay IPv6 Addresses Discovery using Chutney2020-06-27T13:18:34ZteorProp 312: 5.1. Test Relay IPv6 Addresses Discovery using ChutneyTest the IPv6 ORPort Address Detection changes in proposal 312, using chutney networks.
Since chutney runs on private addresses, these tests might be difficult to write, or they might not tell us much about how tor relays behave on the ...Test the IPv6 ORPort Address Detection changes in proposal 312, using chutney networks.
Since chutney runs on private addresses, these tests might be difficult to write, or they might not tell us much about how tor relays behave on the public tor network. So we shouldn't spend much time on them.
See proposal 312, section 5.1:
https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv6-addr.txt#n1461Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33267Prop 313: 8.1. Test IPv6 Relay Consensus Counts using Chutney2020-06-27T13:18:34ZteorProp 313: 8.1. Test IPv6 Relay Consensus Counts using ChutneyTest the IPv6 relay count script in proposal 313, using chutney networks.
Since chutney creates a limited number of relays, we also need to test these changes on consensuses from the public tor network. So we shouldn't spend much time t...Test the IPv6 relay count script in proposal 313, using chutney networks.
Since chutney creates a limited number of relays, we also need to test these changes on consensuses from the public tor network. So we shouldn't spend much time testing the script with chutney.
See proposal 313, section 8.1, chutney tests part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n337Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33271Prop 313: 8.2. Test IPv6 Statistics using Chutney2020-06-27T13:18:33ZteorProp 313: 8.2. Test IPv6 Statistics using ChutneyWe want to test IPv6 connection and consumed bandwidth statistics using
chutney networks. However, chutney runs for a short amount of time, and
creates a limited amount of traffic, so we also need to test these changes
on the public tor ...We want to test IPv6 connection and consumed bandwidth statistics using
chutney networks. However, chutney runs for a short amount of time, and
creates a limited amount of traffic, so we also need to test these changes
on the public tor network.
In particular, we have struggled to test statistics using chutney, because
tor's hard-coded statistics period is 24 hours. (And most chutney networks
run for under 1 minute.)
Maybe we can change the statistics interval to 10-30 seconds, and see if that works?
See proposal 313, section 8.2, chutney tests part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n355Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33302Run bridges+hs-v23 as Chutney's default network2020-06-27T13:18:33ZteorRun bridges+hs-v23 as Chutney's default networkNow that 0.2.9 is obsolete, we can upgrade chutney's defaults.
Users who want to run unsupported tor versions < 0.3.2 can choose a legacy flavour, like `basic-min` or `bridges+hs-v2`.Now that 0.2.9 is obsolete, we can upgrade chutney's defaults.
Users who want to run unsupported tor versions < 0.3.2 can choose a legacy flavour, like `basic-min` or `bridges+hs-v2`.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33304Chutney tries to convert empty pid string to integer2021-09-16T14:47:50ZoparaChutney tries to convert empty pid string to integerIn the `getPid()` function, Chutney reads the pid from a file and converts it to an integer.
```
if not os.path.exists(pidfile):
return None
with open(pidfile, 'r') as f:
return int(f.read())
```
This can result in the followi...In the `getPid()` function, Chutney reads the pid from a file and converts it to an integer.
```
if not os.path.exists(pidfile):
return None
with open(pidfile, 'r') as f:
return int(f.read())
```
This can result in the following error:
```
ValueError: invalid literal for int() with base 10: ''
```
PR incoming...https://gitlab.torproject.org/tpo/core/chutney/-/issues/33333Add a mixed+hs-v23-ipv6 network to chutney2020-06-27T13:18:33ZteorAdd a mixed+hs-v23-ipv6 network to chutneyWe want to test our new IPv6 code in mixed networks, so let's add a `mixed+hs-v23-ipv6` network to chutney.
It should be pretty much a copy-paste from `mixed+hs-v23` and `hs-v23-ipv6-md`.
(There's no need for a non-md variant, all supp...We want to test our new IPv6 code in mixed networks, so let's add a `mixed+hs-v23-ipv6` network to chutney.
It should be pretty much a copy-paste from `mixed+hs-v23` and `hs-v23-ipv6-md`.
(There's no need for a non-md variant, all supported tor versions can use microdescriptors for IPv6 onion services.)
Proposed design:
* 2 dual-stack authorities,
* 2 IPv4-only relays,
* 2 dual-stack relays
* 2 IPv6-only v2 onion services
* 2 IPv6-only v3 onion services
* 2 IPv6-only clients
For each kind of node, one should be a new version, and one should be an old version.
Other networks test IPv4-only authorities and clients, so we don't need them. (But we want to test IPv4 and dual-stack relays in the same network.)Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33353Split chutney's diagnostics into a new script2020-06-27T13:18:33ZteorSplit chutney's diagnostics into a new scriptChutney's failure diagnostics are currently in the Travis CI config file.
But we want to use them in tor's CI. And maybe chutney users want to use them as well.Chutney's failure diagnostics are currently in the Travis CI config file.
But we want to use them in tor's CI. And maybe chutney users want to use them as well.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33376Update the networks in Chutney's CI to match Tor's new test-network*2020-06-27T13:18:33ZteorUpdate the networks in Chutney's CI to match Tor's new test-network*We need to update the list of chutney networks in chutney's CI, to match the networks we added to Tor's `make test-network` and `make test-network-all` in legacy/trac#33334.We need to update the list of chutney networks in chutney's CI, to match the networks we added to Tor's `make test-network` and `make test-network-all` in legacy/trac#33334.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33378Require chutney node bootstrap before running verify2020-06-27T13:18:33ZteorRequire chutney node bootstrap before running verifyAs part of testing relay reachability self-tests in legacy/trac#33232, we need to make sure that all chutney nodes bootstrap.As part of testing relay reachability self-tests in legacy/trac#33232, we need to make sure that all chutney nodes bootstrap.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33379Make chutney wait for all relays in the consensus before verifying2020-06-27T13:18:32ZteorMake chutney wait for all relays in the consensus before verifyingAs part of legacy/trac#33232, we want chutney to check that all relays are in the consensus, then verify.As part of legacy/trac#33232, we want chutney to check that all relays are in the consensus, then verify.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33428Make chutney check for relay microdescriptors before verifying2020-06-27T13:18:32ZteorMake chutney check for relay microdescriptors before verifyingIn legacy/trac#33232, we make chutney check for relay and bridge descriptors, and relays in the consensus, before verifying. But we don't check for relay microdescriptors.
The easiest way to check for relay microdescriptors, is using ea...In legacy/trac#33232, we make chutney check for relay and bridge descriptors, and relays in the consensus, before verifying. But we don't check for relay microdescriptors.
The easiest way to check for relay microdescriptors, is using each relay's ed25519 key. In legacy/trac#30642, we will add an `ed25519-identity` file to each relay's data directory. Chutney can read the contents of this file, and then search for it in `cached-microdescriptors` and `cached-microdescriptors.new`.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33581Restore bridge networkstatus checks in chutney2020-07-21T13:37:22ZteorRestore bridge networkstatus checks in chutneyThis issue depends on the tor bridge descriptor upload fix in legacy/trac#33582, robust reachability self-tests in legacy/trac#33222, or bridge log checks in legacy/trac#34037.
In chutney networks, there's a race condition when bridges...This issue depends on the tor bridge descriptor upload fix in legacy/trac#33582, robust reachability self-tests in legacy/trac#33222, or bridge log checks in legacy/trac#34037.
In chutney networks, there's a race condition when bridges try to publish their descriptor to the bridge authority:
* bridges try to publish their descriptors before bootstrapping
* but bridges can't publish their descriptors, because they don't have enough directory info to build a circuit to the bridge authority
Also, bridges do not retry publishing their descriptor immediately after they bootstrap.
We can only do the networkstatus-bridges check on tor versions with the legacy/trac#33222 or legacy/trac#33582 fixes. So we'll need to check for:
* the next tor 0.4.4-alpha version, or
* an environmental variable that enables these tests.
We don't have to do these fixes, because it should be enough to test relay reachability. But we would risk breaking bridge reachability tests, and not knowing about it until after a release.
Also, the chutney workarounds seem to cause weird race conditions, which are time-consuming to diagnose and fix.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33583Stop setting AssumeReachable on chutney relays and bridges2020-07-23T20:36:05ZteorStop setting AssumeReachable on chutney relays and bridgesWe need to stop setting AssumeReachable on relays and bridges in chutney networks, so they do reachability self-tests.
We should continue to set AssumeReachable on authorities (including the bridge authority), because it disables author...We need to stop setting AssumeReachable on relays and bridges in chutney networks, so they do reachability self-tests.
We should continue to set AssumeReachable on authorities (including the bridge authority), because it disables authority to relay reachability tests. (These tests are still performed on a half-hourly schedule, even in chutney networks. And they are out of scope for Sponsor 55.)
After we implement this change, we should also be able to implement legacy/trac#33581.Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33595Stop waiting for unchecked directory info2020-06-27T13:18:32ZteorStop waiting for unchecked directory infoOnce we've fixed legacy/trac#33428 (microdescs), legacy/trac#33582 (bridges), and legacy/trac#33609 (onion services), we might be able to remove WAIT_FOR_UNCHECKED_DIR_INFO in chutney's TorNet.py.
But we might still need to wait for:
* ...Once we've fixed legacy/trac#33428 (microdescs), legacy/trac#33582 (bridges), and legacy/trac#33609 (onion services), we might be able to remove WAIT_FOR_UNCHECKED_DIR_INFO in chutney's TorNet.py.
But we might still need to wait for:
* internal tor state changes, in response to downloading descriptors.
So we will need to make this change, then test it on Tor 0.3.5 and master.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33596Fix or disable mixed+hs-v2 for Tor 0.3.52020-06-27T13:18:32ZteorFix or disable mixed+hs-v2 for Tor 0.3.5In legacy/trac#33379, we made some changes that broke mixed+hs-v2 on Tor 0.3.5. Strangely, hs-v2-min works fine.
It looks like it is caused by some microdescriptor download bugs that we fixed in 0.4.0 or 0.4.1.
When we finish the rest ...In legacy/trac#33379, we made some changes that broke mixed+hs-v2 on Tor 0.3.5. Strangely, hs-v2-min works fine.
It looks like it is caused by some microdescriptor download bugs that we fixed in 0.4.0 or 0.4.1.
When we finish the rest of legacy/trac#33232, or fix legacy/trac#33428, we should either:
* fix this issue
* require tor or tor-stable to be 0.4.1 or later in mixed+hs-v2Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33597Travis: Revert to basic-min for Tor 0.3.52020-06-27T13:18:32ZteorTravis: Revert to basic-min for Tor 0.3.5In legacy/trac#33379, we changed CI to use single-onion-v23 for 0.3.5, because some other chutney networks were unstable.
Once legacy/trac#33428 or legacy/trac#33583 are fixed, we should be able to revert to basic-min.In legacy/trac#33379, we changed CI to use single-onion-v23 for 0.3.5, because some other chutney networks were unstable.
Once legacy/trac#33428 or legacy/trac#33583 are fixed, we should be able to revert to basic-min.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33604Catch common runtime errors in chutney shell scripts2020-07-23T20:31:12ZteorCatch common runtime errors in chutney shell scriptsLet's gradually convert our scripts to use:
```
set -e
set -u
set -o pipefail
```
And maybe:
```
IFS=$'\n\t'
```
These settings help catch common errors in bash scripts at runtime:
http://redsymbol.net/articles/unofficial-bash-strict-m...Let's gradually convert our scripts to use:
```
set -e
set -u
set -o pipefail
```
And maybe:
```
IFS=$'\n\t'
```
These settings help catch common errors in bash scripts at runtime:
http://redsymbol.net/articles/unofficial-bash-strict-mode/
(Shellcheck helps us catch errors while writing scripts, but it can't help with runtime failures.)
Follow-up to legacy/trac#33451.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33609Check that onion services have successfully posted descriptors before verifying2020-07-07T15:12:56ZteorCheck that onion services have successfully posted descriptors before verifyingBefore verifying, chutney checks that:
* each relay descriptor is cached at each node
* each relay is in a consensus, cached at each node
* each relay is in a microdesc consensus, cached at each node
* each bridge descriptor is cached at...Before verifying, chutney checks that:
* each relay descriptor is cached at each node
* each relay is in a consensus, cached at each node
* each relay is in a microdesc consensus, cached at each node
* each bridge descriptor is cached at each bridge client
We have other tickets for checking:
* microdescriptors
* cached bridge descriptors at the bridge authority
* the bridge networkstatus
That just leaves onion services.
Onion services are tricky, because they post to some HSDirs in the network, but not all. And those HSDirs don't cache the onion service descriptors in a file.
So here is one possible design for this feature:
* check each onion service log for a successful descriptor post to at least one HSDir
* check v2 and v3 onion services
* call it an extra 200% "bootstrap" stage (because it's a sender log check, not a receiver cached file check)
* require 200% bootstrap for onion servicesNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33615Wait for at least 60 seconds for 0.3.5 and earlier to bootstrap2020-06-27T13:18:31ZteorWait for at least 60 seconds for 0.3.5 and earlier to bootstrapThe basic-min network is still a bit unstable, but only for 0.3.5. (And it's much more unrealiable on macOS than Linux.)
It seems like 0.3.5 needs around 60 seconds to bootstrap and sync descriptors. Later versions have microdescriptor ...The basic-min network is still a bit unstable, but only for 0.3.5. (And it's much more unrealiable on macOS than Linux.)
It seems like 0.3.5 needs around 60 seconds to bootstrap and sync descriptors. Later versions have microdescriptor download fixes, and they can bootstrap and verify in just over 20 seconds on basic-min.
So let's make the minimum wait time 70 seconds (3 consensuses), if any nodes in the network are running 0.3.5 (or earlier).Sponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33675Search microdescriptor files for relay ed25519 keys2020-06-27T13:18:31ZteorSearch microdescriptor files for relay ed25519 keysYour code in legacy/trac#33428 needs to pass your local "make test-network-all", before you start this ticket.
We need to enable searching for ed25519 keys in relay microdescriptor files.
There are instructions and a draft search patte...Your code in legacy/trac#33428 needs to pass your local "make test-network-all", before you start this ticket.
We need to enable searching for ed25519 keys in relay microdescriptor files.
There are instructions and a draft search pattern here:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L1325
Please open a new pull request for this ticket. Your branch should be based on the final version of legacy/trac#33428.
Before you push new changes to your pull request, your chutney code should pass:
* "make test-network-all" on tor master
* "make test-network-all" on tor maint-0.3.5
You can build a tor branch using these commands:
```
cd tor
git checkout -b <branch>
make
```
Where <branch> is master or maint-0.3.5https://gitlab.torproject.org/tpo/core/chutney/-/issues/33676Stop waiting a set time for microdescriptors2020-06-27T13:18:31ZteorStop waiting a set time for microdescriptorsYour code in legacy/trac#33675 needs to pass your local "make test-network-all", on master and maint-0.3.5, before you start this ticket.
When we make chutney check for microdescriptors, we can stop waiting a set time for microdescripto...Your code in legacy/trac#33675 needs to pass your local "make test-network-all", on master and maint-0.3.5, before you start this ticket.
When we make chutney check for microdescriptors, we can stop waiting a set time for microdescriptors to download.
Please read these instructions carefully. Don't leave out any steps!
First, set MIN_START_TIME_LEGACY and NODE_WAIT_FOR_UNCHECKED_DIR_INFO to 0:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L926
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L952
Then check that `make test-network-all` still passes on:
* Tor master, and
* Tor maint-0.3.5
You can build a different tor branch using the following commands:
```
git checkout <branch>
make
```
Check the new code on your own machine, and then submit a pull request to run Chutney's CI.
If all those tests pass, you can remove all the code that uses MIN_START_TIME_LEGACY and NODE_WAIT_FOR_UNCHECKED_DIR_INFO:
* remove NODE_WAIT_FOR_UNCHECKED_DIR_INFO
* replace NODE_WAIT_FOR_UNCHECKED_DIR_INFO with 0 in getUncheckedDirInfoWaitTime()
* remove getMinStartTime()
* remove all the constants getMinStartTime() uses
* remove all calls to getMinStartTime()
* remove all the variables, code, and comments that depend on getMinStartTime()
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2224
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2273
Then add a commit to your pull request with these changes, and check that they pass CI.
You can ignore macOS hangs ("red ! symbol") on Travis CI. It is a bit unstable right now.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33793Avoid some race conditions when running chutney networks in series2020-06-27T13:18:30ZteorAvoid some race conditions when running chutney networks in seriesWhen chutney runs multiple networks, one after the others, sometimes older tor instances are left running.
To avoid these issues, chutney can:
* wait longer after terminating tor processes
* cleanup pid files, so we don't accidentally t...When chutney runs multiple networks, one after the others, sometimes older tor instances are left running.
To avoid these issues, chutney can:
* wait longer after terminating tor processes
* cleanup pid files, so we don't accidentally terminate unrelated processesSponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33842chutney: Add network template for OBv3 testing2020-06-27T13:18:30ZGeorge Kadianakischutney: Add network template for OBv3 testinghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33920Remove errors or typos from TorNet.py and Templating.py2020-06-27T13:18:30ZMrSquancheeRemove errors or typos from TorNet.py and Templating.pyI use vscode for programming and when I open chutney files, TorNet.py
and Templating.py, vscode reports a bunch of errors and warnings.
These are often due to typing mistakes and unused variables.
This ticket aims at removing them.I use vscode for programming and when I open chutney files, TorNet.py
and Templating.py, vscode reports a bunch of errors and warnings.
These are often due to typing mistakes and unused variables.
This ticket aims at removing them.MrSquancheeMrSquancheehttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33957Unexpected keyword argument 'bufsize' in subprocess.check_output()2020-06-27T13:18:30ZteorUnexpected keyword argument 'bufsize' in subprocess.check_output()On some python versions, chutney's call to subprocess.check_output() results in the following error:
```
Unexpected keyword argument 'bufsize'…
```
https://github.com/torproject/chutney/pull/68#issuecomment-614596946
The check_output() ...On some python versions, chutney's call to subprocess.check_output() results in the following error:
```
Unexpected keyword argument 'bufsize'…
```
https://github.com/torproject/chutney/pull/68#issuecomment-614596946
The check_output() documentation is unclear about which arguments are supported: it simply says that most arguments to Popen() are supported. (And bufsize is an argument to Popen().):
https://docs.python.org/3.5/library/subprocess.html#subprocess.check_output
https://docs.python.org/3.5/library/subprocess.html#subprocess.Popen
I think our best option here is to remove the bufsize argument, and accept the default buffered behaviour.https://gitlab.torproject.org/tpo/core/chutney/-/issues/34447Chutney networks need at least 3 AssumeReachable nodes.2020-06-27T13:18:29ZNick MathewsonChutney networks need at least 3 AssumeReachable nodes.Unless at least three nodes on a chutney network are AssumeReachable, the consensus as published will not have 3 descriptors in it. Without 3 descriptors in the initial consensus, relays won't be able to build self-testing circuits, dec...Unless at least three nodes on a chutney network are AssumeReachable, the consensus as published will not have 3 descriptors in it. Without 3 descriptors in the initial consensus, relays won't be able to build self-testing circuits, decide they are reachable, and publish their own descriptors... so they will never join the consensus.
Right now, this problem is masked by legacy/trac#34446, which means that we haven't actually disabled AssumeReachable as much as we think we have.Tor: 0.4.5.x-finalhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/34448Chutney templates may need an option-fallback feature2020-06-27T13:18:29ZNick MathewsonChutney templates may need an option-fallback featureIn the parent, we want to split AssumeReachable into AssumeSelfReachable and AssumeOthersReachable. We will want Chutney authorities to set AssumeSelfReachable if that option exists, but AssumeReachable if it does not.
That's not somet...In the parent, we want to split AssumeReachable into AssumeSelfReachable and AssumeOthersReachable. We will want Chutney authorities to set AssumeSelfReachable if that option exists, but AssumeReachable if it does not.
That's not something that the chutney config system can do right now AFAICT.
It might be simpler to do this with a magic bit of code specific to assumereachable as part of the chutney system, but I'm not sure.Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/34449There are too many networks in the chutney networks directory.2020-06-27T13:18:29ZNick MathewsonThere are too many networks in the chutney networks directory.There are 66 "networks" files in the chutney networks directory; of these, I doubt more than a few are actually used for testing things nowadays. I believe we should move aside (or just remove) the ones that are not in current use.
Thi...There are 66 "networks" files in the chutney networks directory; of these, I doubt more than a few are actually used for testing things nowadays. I believe we should move aside (or just remove) the ones that are not in current use.
This came up for me because I will need to make a change to some networks in legacy/trac#34447, and for now it is very hard to tell which of these 66 files I need to make a change for.Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40000Gitlab Migration Milestone2020-06-13T13:32:08ZTracGitlab Migration MilestoneWe're creating this ticket as a part of the Trac-to-Gitlab migration, so that each project's numbering for new tickets will start with 40001.We're creating this ticket as a part of the Trac-to-Gitlab migration, so that each project's numbering for new tickets will start with 40001.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40001Style consideration for shell script `set` directives2020-06-24T02:43:18ZcStyle consideration for shell script `set` directivesTicket #33604 wants shell scripts to be more pedantic. However, looking at current scripts, some use `set -e` and others use the longer verbose `set -o errexit`, so which do we want?
Furthermore there is also the option to set these in ...Ticket #33604 wants shell scripts to be more pedantic. However, looking at current scripts, some use `set -e` and others use the longer verbose `set -o errexit`, so which do we want?
Furthermore there is also the option to set these in the shebang line (i.e. `#!/bin/sh -eu`), which to me is nicer because it's clear that these flags apply to the script. When adding code we definitely don't want someone to place any lines of code above the `set -eu`, so having it in the shebang would prevent that.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40002os.path and pathlib2020-08-02T21:48:20Zcos.path and pathlibThere is a mix of `os.path.join()` calls and simple string concatenation throughout the code. Examples: `getTests()` versus `make_datadir_subdirectory()` in TorNet.py (also, just noticed the mixed camelCase and snake_case which might des...There is a mix of `os.path.join()` calls and simple string concatenation throughout the code. Examples: `getTests()` versus `make_datadir_subdirectory()` in TorNet.py (also, just noticed the mixed camelCase and snake_case which might deserve its own style ticket).
I think we want to be consistent with path handling, but has pathlib been taken into consideration to make it more self-documenting where Chutney needs to deal with paths?cchttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40003Document DEFAULTS options in TorNet.py2020-06-24T14:07:37ZcDocument DEFAULTS options in TorNet.pyThere is a comment that states plainly `# XXX: document these options` so I will go ahead and make a tracking issue for this. I think some of these options are documented in other places in the code already, so that should make the job a...There is a comment that states plainly `# XXX: document these options` so I will go ahead and make a tracking issue for this. I think some of these options are documented in other places in the code already, so that should make the job a bit easier.cchttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40004Chutney tests failing now that AssumeReachable is off.2020-06-26T13:35:27ZNick MathewsonChutney tests failing now that AssumeReachable is off.Since I turned AssumeReachable off by default on relays, several chutney tests have become less reliable. I'll have to investigate why. Do we need a longer timeout?Since I turned AssumeReachable off by default on relays, several chutney tests have become less reliable. I'll have to investigate why. Do we need a longer timeout?Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33230Prop 311: 6.1. Ask Relay Operators to Test IPv6 Reachability2020-08-07T13:16:23ZteorProp 311: 6.1. Ask Relay Operators to Test IPv6 ReachabilitySee legacy/trac#33229 for advice on testing stages.
Write an email to the tor-relays list, asking relay operators to help test the proposal 311 IPv6 ORPort Reachability and Extends changes.
Once these changes are merged, volunteer rela...See legacy/trac#33229 for advice on testing stages.
Write an email to the tor-relays list, asking relay operators to help test the proposal 311 IPv6 ORPort Reachability and Extends changes.
Once these changes are merged, volunteer relay and bridge operators will be able to test them by:
* compiling from source,
* running nightly builds, or
* running alpha releases.
See proposal 311, section 6.1, relay operator test part:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n671Sponsor 55: Improving the Tor network’s IPv6 supportNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40005Chutney Bridges+hs-v23 network is failing in CI2020-07-21T13:39:10ZNick MathewsonChutney Bridges+hs-v23 network is failing in CIIn CI, and on my desktop, chutney bridges+hs-v23 is failing for clients and relays. Apparently, nothing finds that the bridge successfully bootstrapped.
See also related tickets chutney#33581, tor#33582, and tor#33407.
I have a works...In CI, and on my desktop, chutney bridges+hs-v23 is failing for clients and relays. Apparently, nothing finds that the bridge successfully bootstrapped.
See also related tickets chutney#33581, tor#33582, and tor#33407.
I have a works-for-me fix currently in github CI.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33231Prop 311: 6.3. Test Legacy Relays Accept IPv6 Extends using Chutney2020-08-14T12:40:05ZteorProp 311: 6.3. Test Legacy Relays Accept IPv6 Extends using ChutneyThis ticket depends on relay IPv6 extends in legacy/trac#33220. We may also need to disable the "Relay=3" protocol version check.
Test IPv6 extends from newer relays (which implement proposal 311) to older relays (which do not).
Althou...This ticket depends on relay IPv6 extends in legacy/trac#33220. We may also need to disable the "Relay=3" protocol version check.
Test IPv6 extends from newer relays (which implement proposal 311) to older relays (which do not).
Although proposal 311 does not create these kinds of circuits, we need to check for bugs and excessive logs in older tor versions.
See proposal 311, section 6.3:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n698Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33229Prop 311: 6.1. Test IPv6 ORPort Reachability on the Tor Network2020-07-28T15:02:26ZteorProp 311: 6.1. Test IPv6 ORPort Reachability on the Tor NetworkTest the IPv6 ORPort Reachability and Extends changes in proposal 311, on the public tor network, with a small number of relays and bridges.
We can do public tests after each major feature is merged. We can
also wait for multiple featur...Test the IPv6 ORPort Reachability and Extends changes in proposal 311, on the public tor network, with a small number of relays and bridges.
We can do public tests after each major feature is merged. We can
also wait for multiple features to merge, and test them all.
Here are some useful features that we can test:
legacy/trac#33222 / legacy/trac#33226 IPv6 ORPort Reachability Self-Tests / Relay=3 Protocol
legacy/trac#33223 Don't Publish Descriptor if IPv6 ORPort is Unreachable
See proposal 311, section 6.1, initial public test part:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n671Sponsor 55: Improving the Tor network’s IPv6 supportNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40007Our IPv6 networks must have at least 2 IPv6 relays2020-07-21T18:55:46ZDavid Gouletdgoulet@torproject.orgOur IPv6 networks must have at least 2 IPv6 relaysWithout this, the relays in our IPv6 network can NOT self-test properly because they are never able to find a second hop for the self test circuit that supports IPv6 `EXTEND`.Without this, the relays in our IPv6 network can NOT self-test properly because they are never able to find a second hop for the self test circuit that supports IPv6 `EXTEND`.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40008Add a relay-v6.tmpl template file2020-07-22T14:27:22ZDavid Gouletdgoulet@torproject.orgAdd a relay-v6.tmpl template fileFor some reasons, `relay.tmpl` includes `exit-v6.i` which now requires `ipv6_addr` to be set else `None` is put in place.
This makes `basic-min` fails because it ends up with `ORPort None:5001 IPv6Only`. Instead, make `relay.tmpl` to be...For some reasons, `relay.tmpl` includes `exit-v6.i` which now requires `ipv6_addr` to be set else `None` is put in place.
This makes `basic-min` fails because it ends up with `ORPort None:5001 IPv6Only`. Instead, make `relay.tmpl` to be IPv4 only and `relay-v6.tmpl` to be v4 _and_ v6.
This makes all network configuration using `relay.tmpl` to work normally with IPv4 only.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40012Use string, not repr, for log reporting PosixPath2020-09-14T15:56:57ZcUse string, not repr, for log reporting PosixPathFix for #40002 introduced a small formatting issue in Chutney's log output, e.g.:
```
NOTE: creating PosixPath('/home/c/git/tor/../chutney/net/nodes.1596403717'), linking to PosixPath('/home/c/git/tor/../chutney/net/nodes')
```
Easy en...Fix for #40002 introduced a small formatting issue in Chutney's log output, e.g.:
```
NOTE: creating PosixPath('/home/c/git/tor/../chutney/net/nodes.1596403717'), linking to PosixPath('/home/c/git/tor/../chutney/net/nodes')
```
Easy enough fix, I just need to make log output refer to the string expression of `PosixPath`s rather than the representation.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40014Tor 0day: Finding IP Addresses2021-02-02T20:21:12ZtorbugsTor 0day: Finding IP AddressesHi,
*I am new to this GitLab (previous Trac system was much better, no JS!). I hope I am in the right section.
I am writing to report something [which sounds quite alarming] which I found in a web article:
https://www.hackerfactor.com...Hi,
*I am new to this GitLab (previous Trac system was much better, no JS!). I hope I am in the right section.
I am writing to report something [which sounds quite alarming] which I found in a web article:
https://www.hackerfactor.com/blog/index.php?/archives/896-Tor-0day-Finding-IP-Addresses.html
As I am not an expert, I hope someone to take a look a this and hopefully provide feedback of whether it is an actual issue, if there is something to test, if it is fixed etc. Any info is much appreciated.
Thanks.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40015Add HSv3 template only2021-02-16T15:24:31ZDavid Gouletdgoulet@torproject.orgAdd HSv3 template onlyWe have a bunch of networks for `hs-v23` but because HSv2 is going away, we need v3 only tests.We have a bunch of networks for `hs-v23` but because HSv2 is going away, we need v3 only tests.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.org