Chutney issueshttps://gitlab.torproject.org/tpo/core/chutney/-/issues2021-02-16T15:24:31Zhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40015Add HSv3 template only2021-02-16T15:24:31ZDavid Gouletdgoulet@torproject.orgAdd HSv3 template onlyWe have a bunch of networks for `hs-v23` but because HSv2 is going away, we need v3 only tests.We have a bunch of networks for `hs-v23` but because HSv2 is going away, we need v3 only tests.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40014Tor 0day: Finding IP Addresses2021-02-02T20:21:12ZtorbugsTor 0day: Finding IP AddressesHi,
*I am new to this GitLab (previous Trac system was much better, no JS!). I hope I am in the right section.
I am writing to report something [which sounds quite alarming] which I found in a web article:
https://www.hackerfactor.com...Hi,
*I am new to this GitLab (previous Trac system was much better, no JS!). I hope I am in the right section.
I am writing to report something [which sounds quite alarming] which I found in a web article:
https://www.hackerfactor.com/blog/index.php?/archives/896-Tor-0day-Finding-IP-Addresses.html
As I am not an expert, I hope someone to take a look a this and hopefully provide feedback of whether it is an actual issue, if there is something to test, if it is fixed etc. Any info is much appreciated.
Thanks.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40012Use string, not repr, for log reporting PosixPath2020-09-14T15:56:57ZcUse string, not repr, for log reporting PosixPathFix for #40002 introduced a small formatting issue in Chutney's log output, e.g.:
```
NOTE: creating PosixPath('/home/c/git/tor/../chutney/net/nodes.1596403717'), linking to PosixPath('/home/c/git/tor/../chutney/net/nodes')
```
Easy en...Fix for #40002 introduced a small formatting issue in Chutney's log output, e.g.:
```
NOTE: creating PosixPath('/home/c/git/tor/../chutney/net/nodes.1596403717'), linking to PosixPath('/home/c/git/tor/../chutney/net/nodes')
```
Easy enough fix, I just need to make log output refer to the string expression of `PosixPath`s rather than the representation.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40008Add a relay-v6.tmpl template file2020-07-22T14:27:22ZDavid Gouletdgoulet@torproject.orgAdd a relay-v6.tmpl template fileFor some reasons, `relay.tmpl` includes `exit-v6.i` which now requires `ipv6_addr` to be set else `None` is put in place.
This makes `basic-min` fails because it ends up with `ORPort None:5001 IPv6Only`. Instead, make `relay.tmpl` to be...For some reasons, `relay.tmpl` includes `exit-v6.i` which now requires `ipv6_addr` to be set else `None` is put in place.
This makes `basic-min` fails because it ends up with `ORPort None:5001 IPv6Only`. Instead, make `relay.tmpl` to be IPv4 only and `relay-v6.tmpl` to be v4 _and_ v6.
This makes all network configuration using `relay.tmpl` to work normally with IPv4 only.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40007Our IPv6 networks must have at least 2 IPv6 relays2020-07-21T18:55:46ZDavid Gouletdgoulet@torproject.orgOur IPv6 networks must have at least 2 IPv6 relaysWithout this, the relays in our IPv6 network can NOT self-test properly because they are never able to find a second hop for the self test circuit that supports IPv6 `EXTEND`.Without this, the relays in our IPv6 network can NOT self-test properly because they are never able to find a second hop for the self test circuit that supports IPv6 `EXTEND`.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40005Chutney Bridges+hs-v23 network is failing in CI2020-07-21T13:39:10ZNick MathewsonChutney Bridges+hs-v23 network is failing in CIIn CI, and on my desktop, chutney bridges+hs-v23 is failing for clients and relays. Apparently, nothing finds that the bridge successfully bootstrapped.
See also related tickets chutney#33581, tor#33582, and tor#33407.
I have a works...In CI, and on my desktop, chutney bridges+hs-v23 is failing for clients and relays. Apparently, nothing finds that the bridge successfully bootstrapped.
See also related tickets chutney#33581, tor#33582, and tor#33407.
I have a works-for-me fix currently in github CI.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40004Chutney tests failing now that AssumeReachable is off.2020-06-26T13:35:27ZNick MathewsonChutney tests failing now that AssumeReachable is off.Since I turned AssumeReachable off by default on relays, several chutney tests have become less reliable. I'll have to investigate why. Do we need a longer timeout?Since I turned AssumeReachable off by default on relays, several chutney tests have become less reliable. I'll have to investigate why. Do we need a longer timeout?Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40003Document DEFAULTS options in TorNet.py2020-06-24T14:07:37ZcDocument DEFAULTS options in TorNet.pyThere is a comment that states plainly `# XXX: document these options` so I will go ahead and make a tracking issue for this. I think some of these options are documented in other places in the code already, so that should make the job a...There is a comment that states plainly `# XXX: document these options` so I will go ahead and make a tracking issue for this. I think some of these options are documented in other places in the code already, so that should make the job a bit easier.cchttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40002os.path and pathlib2020-08-02T21:48:20Zcos.path and pathlibThere is a mix of `os.path.join()` calls and simple string concatenation throughout the code. Examples: `getTests()` versus `make_datadir_subdirectory()` in TorNet.py (also, just noticed the mixed camelCase and snake_case which might des...There is a mix of `os.path.join()` calls and simple string concatenation throughout the code. Examples: `getTests()` versus `make_datadir_subdirectory()` in TorNet.py (also, just noticed the mixed camelCase and snake_case which might deserve its own style ticket).
I think we want to be consistent with path handling, but has pathlib been taken into consideration to make it more self-documenting where Chutney needs to deal with paths?cchttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40001Style consideration for shell script `set` directives2020-06-24T02:43:18ZcStyle consideration for shell script `set` directivesTicket #33604 wants shell scripts to be more pedantic. However, looking at current scripts, some use `set -e` and others use the longer verbose `set -o errexit`, so which do we want?
Furthermore there is also the option to set these in ...Ticket #33604 wants shell scripts to be more pedantic. However, looking at current scripts, some use `set -e` and others use the longer verbose `set -o errexit`, so which do we want?
Furthermore there is also the option to set these in the shebang line (i.e. `#!/bin/sh -eu`), which to me is nicer because it's clear that these flags apply to the script. When adding code we definitely don't want someone to place any lines of code above the `set -eu`, so having it in the shebang would prevent that.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40000Gitlab Migration Milestone2020-06-13T13:32:08ZTracGitlab Migration MilestoneWe're creating this ticket as a part of the Trac-to-Gitlab migration, so that each project's numbering for new tickets will start with 40001.We're creating this ticket as a part of the Trac-to-Gitlab migration, so that each project's numbering for new tickets will start with 40001.https://gitlab.torproject.org/tpo/core/chutney/-/issues/34449There are too many networks in the chutney networks directory.2020-06-27T13:18:29ZNick MathewsonThere are too many networks in the chutney networks directory.There are 66 "networks" files in the chutney networks directory; of these, I doubt more than a few are actually used for testing things nowadays. I believe we should move aside (or just remove) the ones that are not in current use.
Thi...There are 66 "networks" files in the chutney networks directory; of these, I doubt more than a few are actually used for testing things nowadays. I believe we should move aside (or just remove) the ones that are not in current use.
This came up for me because I will need to make a change to some networks in legacy/trac#34447, and for now it is very hard to tell which of these 66 files I need to make a change for.Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/34448Chutney templates may need an option-fallback feature2020-06-27T13:18:29ZNick MathewsonChutney templates may need an option-fallback featureIn the parent, we want to split AssumeReachable into AssumeSelfReachable and AssumeOthersReachable. We will want Chutney authorities to set AssumeSelfReachable if that option exists, but AssumeReachable if it does not.
That's not somet...In the parent, we want to split AssumeReachable into AssumeSelfReachable and AssumeOthersReachable. We will want Chutney authorities to set AssumeSelfReachable if that option exists, but AssumeReachable if it does not.
That's not something that the chutney config system can do right now AFAICT.
It might be simpler to do this with a magic bit of code specific to assumereachable as part of the chutney system, but I'm not sure.Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/34447Chutney networks need at least 3 AssumeReachable nodes.2020-06-27T13:18:29ZNick MathewsonChutney networks need at least 3 AssumeReachable nodes.Unless at least three nodes on a chutney network are AssumeReachable, the consensus as published will not have 3 descriptors in it. Without 3 descriptors in the initial consensus, relays won't be able to build self-testing circuits, dec...Unless at least three nodes on a chutney network are AssumeReachable, the consensus as published will not have 3 descriptors in it. Without 3 descriptors in the initial consensus, relays won't be able to build self-testing circuits, decide they are reachable, and publish their own descriptors... so they will never join the consensus.
Right now, this problem is masked by legacy/trac#34446, which means that we haven't actually disabled AssumeReachable as much as we think we have.Tor: 0.4.5.x-finalhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33957Unexpected keyword argument 'bufsize' in subprocess.check_output()2020-06-27T13:18:30ZteorUnexpected keyword argument 'bufsize' in subprocess.check_output()On some python versions, chutney's call to subprocess.check_output() results in the following error:
```
Unexpected keyword argument 'bufsize'…
```
https://github.com/torproject/chutney/pull/68#issuecomment-614596946
The check_output() ...On some python versions, chutney's call to subprocess.check_output() results in the following error:
```
Unexpected keyword argument 'bufsize'…
```
https://github.com/torproject/chutney/pull/68#issuecomment-614596946
The check_output() documentation is unclear about which arguments are supported: it simply says that most arguments to Popen() are supported. (And bufsize is an argument to Popen().):
https://docs.python.org/3.5/library/subprocess.html#subprocess.check_output
https://docs.python.org/3.5/library/subprocess.html#subprocess.Popen
I think our best option here is to remove the bufsize argument, and accept the default buffered behaviour.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33920Remove errors or typos from TorNet.py and Templating.py2020-06-27T13:18:30ZMrSquancheeRemove errors or typos from TorNet.py and Templating.pyI use vscode for programming and when I open chutney files, TorNet.py
and Templating.py, vscode reports a bunch of errors and warnings.
These are often due to typing mistakes and unused variables.
This ticket aims at removing them.I use vscode for programming and when I open chutney files, TorNet.py
and Templating.py, vscode reports a bunch of errors and warnings.
These are often due to typing mistakes and unused variables.
This ticket aims at removing them.MrSquancheeMrSquancheehttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33842chutney: Add network template for OBv3 testing2020-06-27T13:18:30ZGeorge Kadianakischutney: Add network template for OBv3 testinghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33793Avoid some race conditions when running chutney networks in series2020-06-27T13:18:30ZteorAvoid some race conditions when running chutney networks in seriesWhen chutney runs multiple networks, one after the others, sometimes older tor instances are left running.
To avoid these issues, chutney can:
* wait longer after terminating tor processes
* cleanup pid files, so we don't accidentally t...When chutney runs multiple networks, one after the others, sometimes older tor instances are left running.
To avoid these issues, chutney can:
* wait longer after terminating tor processes
* cleanup pid files, so we don't accidentally terminate unrelated processesSponsor 55: Improving the Tor network’s IPv6 supportteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/33676Stop waiting a set time for microdescriptors2020-06-27T13:18:31ZteorStop waiting a set time for microdescriptorsYour code in legacy/trac#33675 needs to pass your local "make test-network-all", on master and maint-0.3.5, before you start this ticket.
When we make chutney check for microdescriptors, we can stop waiting a set time for microdescripto...Your code in legacy/trac#33675 needs to pass your local "make test-network-all", on master and maint-0.3.5, before you start this ticket.
When we make chutney check for microdescriptors, we can stop waiting a set time for microdescriptors to download.
Please read these instructions carefully. Don't leave out any steps!
First, set MIN_START_TIME_LEGACY and NODE_WAIT_FOR_UNCHECKED_DIR_INFO to 0:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L926
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L952
Then check that `make test-network-all` still passes on:
* Tor master, and
* Tor maint-0.3.5
You can build a different tor branch using the following commands:
```
git checkout <branch>
make
```
Check the new code on your own machine, and then submit a pull request to run Chutney's CI.
If all those tests pass, you can remove all the code that uses MIN_START_TIME_LEGACY and NODE_WAIT_FOR_UNCHECKED_DIR_INFO:
* remove NODE_WAIT_FOR_UNCHECKED_DIR_INFO
* replace NODE_WAIT_FOR_UNCHECKED_DIR_INFO with 0 in getUncheckedDirInfoWaitTime()
* remove getMinStartTime()
* remove all the constants getMinStartTime() uses
* remove all calls to getMinStartTime()
* remove all the variables, code, and comments that depend on getMinStartTime()
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2224
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2273
Then add a commit to your pull request with these changes, and check that they pass CI.
You can ignore macOS hangs ("red ! symbol") on Travis CI. It is a bit unstable right now.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33675Search microdescriptor files for relay ed25519 keys2020-06-27T13:18:31ZteorSearch microdescriptor files for relay ed25519 keysYour code in legacy/trac#33428 needs to pass your local "make test-network-all", before you start this ticket.
We need to enable searching for ed25519 keys in relay microdescriptor files.
There are instructions and a draft search patte...Your code in legacy/trac#33428 needs to pass your local "make test-network-all", before you start this ticket.
We need to enable searching for ed25519 keys in relay microdescriptor files.
There are instructions and a draft search pattern here:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L1325
Please open a new pull request for this ticket. Your branch should be based on the final version of legacy/trac#33428.
Before you push new changes to your pull request, your chutney code should pass:
* "make test-network-all" on tor master
* "make test-network-all" on tor maint-0.3.5
You can build a tor branch using these commands:
```
cd tor
git checkout -b <branch>
make
```
Where <branch> is master or maint-0.3.5