Chutney issueshttps://gitlab.torproject.org/tpo/core/chutney/-/issues2023-04-12T14:42:09Zhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40021cgitb is deprecated2023-04-12T14:42:09ZNick Mathewsoncgitb is deprecatedWe use the cgitb module to generate nice tracebacks. But it seems that it's been deprecated for ages, and will soon go away in the next version of python3. We should remove or replace it.We use the cgitb module to generate nice tracebacks. But it seems that it's been deprecated for ages, and will soon go away in the next version of python3. We should remove or replace it.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40020Not found error2022-07-25T03:24:47ZPulkit ChandelNot found errorFileNotFoundError: [Errno 2] No such file or directory: '/home/eict/chutney/net/nodes.1657278323/006r/torrc'
</pre>
Why am I Getting this error. The file 006r was never made when I executed tor chutney command, then why is it looking for...FileNotFoundError: [Errno 2] No such file or directory: '/home/eict/chutney/net/nodes.1657278323/006r/torrc'
</pre>
Why am I Getting this error. The file 006r was never made when I executed tor chutney command, then why is it looking for it.And how to solve ithttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40019Rewrite network testing without asyncore/asynchat2023-12-06T09:12:07ZAlex XuRewrite network testing without asyncore/asynchatRunning `python3 -X dev -m chutney.TorNet` with Python 3.11 prints:
```
/tmp/chutney/lib/chutney/Traffic.py:33: DeprecationWarning: The asyncore module is deprecated and will be removed in Python 3.12. The recommended replacement is asy...Running `python3 -X dev -m chutney.TorNet` with Python 3.11 prints:
```
/tmp/chutney/lib/chutney/Traffic.py:33: DeprecationWarning: The asyncore module is deprecated and will be removed in Python 3.12. The recommended replacement is asyncio
import asyncore
/tmp/chutney/lib/chutney/Traffic.py:34: DeprecationWarning: The asynchat module is deprecated and will be removed in Python 3.12. The recommended replacement is asyncio
import asynchat
```
According to https://peps.python.org/pep-0693/, Python 3.12 is scheduled for release on 2023-10-02.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40018chutney can generate invalid arti config with empty ed_identity2022-10-24T21:02:18ZIan Jacksoniwj@torproject.orgchutney can generate invalid arti config with empty ed_identityWhen I run arti's chutney tests on my laptop, chutney generates an `arti.toml` containing something like this:
```
fallback_caches = [
{rsa_identity = "30BB3F0559E82F9F22670F74B2B579618BCB052F", ed_identity = "", orports = ["127.0.0....When I run arti's chutney tests on my laptop, chutney generates an `arti.toml` containing something like this:
```
fallback_caches = [
{rsa_identity = "30BB3F0559E82F9F22670F74B2B579618BCB052F", ed_identity = "", orports = ["127.0.0.1:5000"]},
{rsa_identity = "3EDBDF94A19EACB6CBF33CED2A8A2F2D9F2C71B9", ed_identity = "", orports = ["127.0.0.1:5001"]},
{rsa_identity = "F159C273C9984FC9BC78E3D5B92EA6B43C6F80B2", ed_identity = "", orports = ["127.0.0.1:5002"]},
]
```
arti refusese to load this because the empty string is not a valid ed identity.
I remember that I managed to get this working the last time this happened to me but I can't remember what I did. There don't seem to be any relevant error messages.
The script in question is here:
https://gitlab.torproject.org/tpo/core/arti/-/blob/main/tests/chutney/setuphttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40017Chutney failure when CHUTNEY_DATA_BYTES is large2022-02-07T19:31:44ZAlexander Færøyahf@torproject.orgChutney failure when CHUTNEY_DATA_BYTES is largeMike ran into an issue today where Chutney fails with `all_done` being `False` (see status in `DEBUG: Done with run(); all_done == False and failure_count == 0`). The issue seemed to be related to having set the env variable `CHUTNEY_DAT...Mike ran into an issue today where Chutney fails with `all_done` being `False` (see status in `DEBUG: Done with run(); all_done == False and failure_count == 0`). The issue seemed to be related to having set the env variable `CHUTNEY_DATA_BYTES` to a large value (in this case: 104857600)https://gitlab.torproject.org/tpo/core/chutney/-/issues/40016Improve a way to check that a network has bootstrapped2022-02-07T19:31:43ZjugaImprove a way to check that a network has bootstrappedFor integration tests is very useful to know programmatically when a network has bootstrapped before start using it.
I used to use a similar script to https://github.com/TheTorProject/bwscanner/blob/develop/test/scripts/install-chutney....For integration tests is very useful to know programmatically when a network has bootstrapped before start using it.
I used to use a similar script to https://github.com/TheTorProject/bwscanner/blob/develop/test/scripts/install-chutney.sh#L15, which uses `verify` command but that stop working (don't have now the trace).
Then i realized that there was `wait_for_bootstrap`, but in Gitlab CI, sometimes there're relays missing, using the client and stem.
@nickm has suggested `status`, but it'd need more work to parse its exit.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40013Template relay-non-exit.tmpl does generate exits2022-03-17T11:19:57ZjugaTemplate relay-non-exit.tmpl does generate exitsI assume this is the template to use to generate non-exits, if not this is probably not an issue.
The network `hs-v2-min` uses this template.
I start the network and try to obtain the relays without the EXIT flag:
```
from stem import F...I assume this is the template to use to generate non-exits, if not this is probably not an issue.
The network `hs-v2-min` uses this template.
I start the network and try to obtain the relays without the EXIT flag:
```
from stem import Flag
from stem.control import Controller
c = Controller.from_port(port=8000)
c.authenticate()
cons = c.get_network_statuses()
[(rs.nickname, rs.flags) for rs in cons if Flag.EXIT not in rs.flags]
>>> []
```
But all relays have the exit flag.
The generated `torrc` has `ExitRelay 0`, so i guess that it must be some other option that is making the relays to be flagged as exits?
Is there other way to generate non-exits?https://gitlab.torproject.org/tpo/core/chutney/-/issues/40011Chutney: remove support for Tor's pre-0.3.5 directory structure2022-02-07T19:31:40ZteorChutney: remove support for Tor's pre-0.3.5 directory structuretest-network*.sh still supports Tor's pre-0.3.5 directory structure. And there may be other Tor 0.2.9 code in chutney.
Now that 0.2.9 is unsupported, we can remove this code.
We should be able to find it by grepping for "0.2.9", "029",...test-network*.sh still supports Tor's pre-0.3.5 directory structure. And there may be other Tor 0.2.9 code in chutney.
Now that 0.2.9 is unsupported, we can remove this code.
We should be able to find it by grepping for "0.2.9", "029", "src/or", "src/common", "0.3.5", or "035".https://gitlab.torproject.org/tpo/core/chutney/-/issues/40010Create a chutney network with a single authority, and make sure it bootstraps2022-02-07T19:31:38ZteorCreate a chutney network with a single authority, and make sure it bootstrapsDepends on legacy/trac#28203, which logs bootstrap progress.
It looks like there are a lot of race conditions in chutney and tor. We can eliminate most of these race conditions by launching a single authority, and seeing if it bootstrap...Depends on legacy/trac#28203, which logs bootstrap progress.
It looks like there are a lot of race conditions in chutney and tor. We can eliminate most of these race conditions by launching a single authority, and seeing if it bootstraps reliably. Then we can narrow down the source of chutney's instability.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/40009Add a stub geoip file for chutney networks2022-02-07T19:31:37ZNick MathewsonAdd a stub geoip file for chutney networksFor Chutney networks, it would be handy to have geoip files for IPv4 and IPv6. Since all chutney relays are on localhost in our networks, the geoip files should map localhost to a make-believe country code.For Chutney networks, it would be handy to have geoip files for IPv4 and IPv6. Since all chutney relays are on localhost in our networks, the geoip files should map localhost to a make-believe country code.https://gitlab.torproject.org/tpo/core/chutney/-/issues/40006Chutney's logic for bootstrap detection has gotten pretty complex2021-09-16T14:47:26ZNick MathewsonChutney's logic for bootstrap detection has gotten pretty complexI have a hard time understanding the code that checks whether directory information is bootstrapped. I think it might be time to refactor it. It all seems to get called via `getNodeDirInfoStatus()`, but after that it gets pretty hairy.I have a hard time understanding the code that checks whether directory information is bootstrapped. I think it might be time to refactor it. It all seems to get called via `getNodeDirInfoStatus()`, but after that it gets pretty hairy.https://gitlab.torproject.org/tpo/core/chutney/-/issues/34037Make chutney check tor's logs for reachability self-test success2020-10-29T15:13:54ZteorMake chutney check tor's logs for reachability self-test successThis ticket is an alternative to legacy/trac#33582 or legacy/trac#33222.
Instead of fixing bridge descriptor uploads, we can check bridge logs to make sure that reachability self-tests have succeeded.
For consistency, we should also do...This ticket is an alternative to legacy/trac#33582 or legacy/trac#33222.
Instead of fixing bridge descriptor uploads, we can check bridge logs to make sure that reachability self-tests have succeeded.
For consistency, we should also do the same checks for relays.
We can only do these tests on authorities, relays, and bridges that are configured with `AssumeReachable 0`. Chutney's current defaults are:
* directory authorities: 1
* bridge authorities: 1
* relays: 0
* bridges: 0
* clients: clients never perform reachability self-tests
Some custom chutney networks may set `AssumeReachable 1` for relays and bridges. So we should make it easy for them to disable these checks.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33825Make Environ handle "in" and "get()" like a dict2022-02-07T19:31:36ZteorMake Environ handle "in" and "get()" like a dictSome standard Python dict code doesn't work on chutney's Environ class:
```
is_in_env = 'foo' in self._env
value_or_none = self._env.get('foo')
```
"in" should return a boolean, and "get()" should return the value (or None).
But instea...Some standard Python dict code doesn't work on chutney's Environ class:
```
is_in_env = 'foo' in self._env
value_or_none = self._env.get('foo')
```
"in" should return a boolean, and "get()" should return the value (or None).
But instead, when the key is missing, sometimes they throw a KeyError. (It seems to happen in certain contexts, but not others.)
We should work out if Environ is missing some of the required dict implementation functions. Or if there is some issue with Environ's lookup code.
Then we should implement unit tests, to make sure we don't break Environ in future.https://gitlab.torproject.org/tpo/core/chutney/-/issues/33677Stop waiting a set time for onion service descriptors2022-02-07T19:31:34ZteorStop waiting a set time for onion service descriptorsAfter we make chutney check for onion service descriptors in legacy/trac#33609, we can stop waiting a set time for onion service descriptors to download.
First, set HS_WAIT_FOR_UNCHECKED_DIR_INFO to 0:
https://github.com/torproject/chut...After we make chutney check for onion service descriptors in legacy/trac#33609, we can stop waiting a set time for onion service descriptors to download.
First, set HS_WAIT_FOR_UNCHECKED_DIR_INFO to 0:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L953
Then check that `make test-network-all` still passes on:
* Tor master,
* Tor maint-0.3.5
Then check that chutney's CI still passes.
If all those tests pass, we can remove all the code that uses HS_WAIT_FOR_UNCHECKED_DIR_INFO:
* remove HS_WAIT_FOR_UNCHECKED_DIR_INFO
* remove getUncheckedDirInfoWaitTime()
* remove all the variables, code, and comments that depend on getUncheckedDirInfoWaitTime()
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2226
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2266https://gitlab.torproject.org/tpo/core/chutney/-/issues/33598chutney does not fail on some SOCKS errors2022-02-07T19:32:13Zteorchutney does not fail on some SOCKS errorsWhen tor can't make a connection, and sends back a SOCKS error, chutney keeps on rapidly sending SOCKS requests. Instead, chutney should fail.
I think we introduced this bug when we started using asyncore.
I have worked around the bug...When tor can't make a connection, and sends back a SOCKS error, chutney keeps on rapidly sending SOCKS requests. Instead, chutney should fail.
I think we introduced this bug when we started using asyncore.
I have worked around the bug using a 5 second asyncore timeout, but we should come up with a permanent fix.
I think nickm might be able to help with this issue, because he wrote that code.https://gitlab.torproject.org/tpo/core/chutney/-/issues/31833Make chutney fail if Tor logs a bug warning2022-02-07T19:31:30ZteorMake chutney fail if Tor logs a bug warningWe've missed bug warnings in our CI, because chutney succeeds even if Tor logs a bug warning.
For example legacy/trac#31793.We've missed bug warnings in our CI, because chutney succeeds even if Tor logs a bug warning.
For example legacy/trac#31793.https://gitlab.torproject.org/tpo/core/chutney/-/issues/31639When Travis updates the homebrew cache in their images, stop updating it in ....2022-02-07T19:31:27ZteorWhen Travis updates the homebrew cache in their images, stop updating it in .travis.ymlIn legacy/trac#30928, we added a homebrew update to .travis.yml for shellcheck.
legacy/trac#30279 also uses an updated tor for the IPv6 v3 single onion service job.
This ticket removes that update.In legacy/trac#30928, we added a homebrew update to .travis.yml for shellcheck.
legacy/trac#30279 also uses an updated tor for the IPv6 v3 single onion service job.
This ticket removes that update.https://gitlab.torproject.org/tpo/core/chutney/-/issues/30886Add pluggable transport support to chutney's tools/test-network.sh2022-02-07T19:31:25ZteorAdd pluggable transport support to chutney's tools/test-network.shWe have basic PT support in chutney, but we need a nicer interface to it.We have basic PT support in chutney, but we need a nicer interface to it.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30884Test pluggable transports in chutney's CI2022-02-07T19:30:54ZteorTest pluggable transports in chutney's CIWe added PT support to chutney in legacy/trac#29024, now we need to make sure it keeps on working.We added PT support to chutney in legacy/trac#29024, now we need to make sure it keeps on working.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/30720Use test-network.sh for the examples in Chutney's README2022-02-07T19:30:52ZteorUse test-network.sh for the examples in Chutney's READMEChutney's README is confusing, because we provide examples that assume you have already started a chutney network.
Instead, we should provide test-network.sh command lines.Chutney's README is confusing, because we provide examples that assume you have already started a chutney network.
Instead, we should provide test-network.sh command lines.