Chutney issueshttps://gitlab.torproject.org/tpo/core/chutney/-/issues2020-06-27T13:19:08Zhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/6413Add support for bridges and clients on IPv6 to chutney2020-06-27T13:19:08ZLinus Nordberglinus@torproject.orgAdd support for bridges and clients on IPv6 to chutneyBranch ipv6-bridges in user/linus/chutney.git contains support for
bridges on IPv6 and clients using them.Branch ipv6-bridges in user/linus/chutney.git contains support for
bridges on IPv6 and clients using them.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/6753chutney needs to set AlternateBridgeAuthority too2020-06-27T13:19:08ZRoger Dingledinechutney needs to set AlternateBridgeAuthority tooSince git commit 21c6c8485 (to Tor), Tor won't start if TestingTorNetwork is on, AlternateDirAuthority is set, but AlternateBridgeAuthority is not.Since git commit 21c6c8485 (to Tor), Tor won't start if TestingTorNetwork is on, AlternateDirAuthority is set, but AlternateBridgeAuthority is not.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/6754chutney traceback when it can't kill its tors fast enough2020-06-27T13:19:07ZRoger Dingledinechutney traceback when it can't kill its tors fast enough```
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globa...```
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/arma/old/torgit/chutney/lib/chutney/TorNet.py", line 663, in <module>
main()
File "/home/arma/old/torgit/chutney/lib/chutney/TorNet.py", line 660, in main
runConfigFile(sys.argv[1], f)
File "/home/arma/old/torgit/chutney/lib/chutney/TorNet.py", line 646, in runConfigFile
getattr(network,verb)()
File "/home/arma/old/torgit/chutney/lib/chutney/TorNet.py", line 619, in stop
n.check(listNonRunning=False)
AttributeError: 'int' object has no attribute 'check'
```Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/8531Extend Chutney to be able to verify that we can send and retrieve traffic2020-06-27T13:19:07ZLinus Nordberglinus@torproject.orgExtend Chutney to be able to verify that we can send and retrieve trafficThis is related to legacy/trac#8530.This is related to legacy/trac#8530.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/9086Rewrite network testing using asyncore/asynchat2020-07-17T11:01:53ZLinus Nordberglinus@torproject.orgRewrite network testing using asyncore/asynchatAs Nick points out in legacy/trac#8531, we should use asyncore or maybe even
asynchat for the network testing in Traffic.py.
This will give us more readable code and higher code quality.As Nick points out in legacy/trac#8531, we should use asyncore or maybe even
asynchat for the network testing in Traffic.py.
This will give us more readable code and higher code quality.https://gitlab.torproject.org/tpo/core/chutney/-/issues/9087Move network tests out of TorNet.py2020-06-27T13:19:07ZLinus Nordberglinus@torproject.orgMove network tests out of TorNet.pyAs Nick points out in legacy/trac#8531, it's far from perfect that new network
tests need to go into TorNet.py.
Maybe read a config file or pick up .py files from a tests/ directory?As Nick points out in legacy/trac#8531, it's far from perfect that new network
tests need to go into TorNet.py.
Maybe read a config file or pick up .py files from a tests/ directory?https://gitlab.torproject.org/tpo/core/chutney/-/issues/10136Unknown options in torrc file2020-06-27T13:19:07ZTracUnknown options in torrc fileI am new(student) to tor. When I try to run chutney => ./chutney start networks/basic , it is not able to run the tor. When I looked at the torrc file generated and tried running them separately, I got the following warnings:
* Unknown ...I am new(student) to tor. When I try to run chutney => ./chutney start networks/basic , it is not able to run the tor. When I looked at the torrc file generated and tried running them separately, I got the following warnings:
* Unknown option 'TestingClientDownloadSchedule'
* Unknown option 'TestingV3AuthVotingStartOffset'
with this I also got(after commenting above lines from torrc file) :
* TestingV3AuthInitialVoteDelay is way too low
**Trac**:
**Username**: vikassyNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/10154Allow a Bridge Authority in Test Networks2020-06-27T13:19:07ZMatthew FinkelAllow a Bridge Authority in Test NetworksCurrently Chutney generates an invalid torrc when the network contains a bridge authority.
```
Nov 13 23:01:36.926 [warn] You cannot set both DirAuthority and Alternate*Authority.
Nov 13 23:01:36.926 [warn] Failed to parse/validate conf...Currently Chutney generates an invalid torrc when the network contains a bridge authority.
```
Nov 13 23:01:36.926 [warn] You cannot set both DirAuthority and Alternate*Authority.
Nov 13 23:01:36.926 [warn] Failed to parse/validate config: Directory authority/fallback line did not parse. See logs for details.
Nov 13 23:01:36.926 [err] Reading config failed--see warnings above.
```Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/10155Allow delayed node start2020-07-17T11:02:21ZMatthew FinkelAllow delayed node startIt will be useful to allow nodes to be started after a certain amount of time, especially when interested in its effect on the consensus.It will be useful to allow nodes to be started after a certain amount of time, especially when interested in its effect on the consensus.https://gitlab.torproject.org/tpo/core/chutney/-/issues/10156Replace DirServer with DirAuthority in torrc2020-07-23T20:05:23ZMatthew FinkelReplace DirServer with DirAuthority in torrcShould the generated torrc use DirAuthority instead of DirServer now that legacy/trac#6791 and legacy/trac#10124 are merged into Tor? It'll probably be smart to have an option to tell Chutney to use DirServer if one tests with an older v...Should the generated torrc use DirAuthority instead of DirServer now that legacy/trac#6791 and legacy/trac#10124 are merged into Tor? It'll probably be smart to have an option to tell Chutney to use DirServer if one tests with an older version.https://gitlab.torproject.org/tpo/core/chutney/-/issues/10764chutney has a non-obvious license2020-06-27T13:19:07ZRoger Dingledinechutney has a non-obvious licenseAt the top of some of the python files in Chutney is this text:
```
# Copyright 2011 Nick Mathewson, Michael Stone
# Copyright 2013 The Tor Project
#
# You may do anything with this work that copyright law would normally
# restrict, so...At the top of some of the python files in Chutney is this text:
```
# Copyright 2011 Nick Mathewson, Michael Stone
# Copyright 2013 The Tor Project
#
# You may do anything with this work that copyright law would normally
# restrict, so long as you retain the above notice(s) and this license
# in all redistributed copies and derived works. There is no warranty.
```
which some folks might recognize as the Tiny license.
But the work as a whole appears to have no license file. So e.g. tools/bootstrap-network.sh would sure look to be non open source software currently.
Can we add a license for the work as a whole?Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/10933support for setting binary paths via environment varaibles2020-06-27T13:19:06ZTracsupport for setting binary paths via environment varaiblesMy branch:
https://github.com/houqp/chutney/tree/env_bin
This makes it easier to switch between different builds for testing purpose.
**Trac**:
**Username**: dave2008My branch:
https://github.com/houqp/chutney/tree/env_bin
This makes it easier to switch between different builds for testing purpose.
**Trac**:
**Username**: dave2008Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/10934remove stale lock file for crashed Tor2020-06-27T13:19:06ZTracremove stale lock file for crashed TorOn stop command, stale lock files will be checked and removed.
branch:
https://github.com/houqp/chutney/commits/remove_stale_lockfile
**Trac**:
**Username**: dave2008On stop command, stale lock files will be checked and removed.
branch:
https://github.com/houqp/chutney/commits/remove_stale_lockfile
**Trac**:
**Username**: dave2008Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/11340cleaner error ouput for none exist network file2020-06-27T13:19:06ZTraccleaner error ouput for none exist network filebranch:
https://github.com/houqp/chutney/tree/miss_network_error
**Trac**:
**Username**: dave2008branch:
https://github.com/houqp/chutney/tree/miss_network_error
**Trac**:
**Username**: dave2008Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/11374fix tor lockfile checking bug2020-07-23T20:06:47ZTracfix tor lockfile checking bugI previously had a misconception that lockfile got removed every time tor exits :(
This patch checks the state of lockfile properly.
branch:
https://github.com/houqp/chutney/tree/lock_fix
**Trac**:
**Username**: dave2008I previously had a misconception that lockfile got removed every time tor exits :(
This patch checks the state of lockfile properly.
branch:
https://github.com/houqp/chutney/tree/lock_fix
**Trac**:
**Username**: dave2008https://gitlab.torproject.org/tpo/core/chutney/-/issues/13161[patch] getting chutney working with tor 0.2.6-alpha on OS X2020-06-27T13:19:06Zteor[patch] getting chutney working with tor 0.2.6-alpha on OS XI had several issues getting chutney working on OS X 10.9 with tor 0.2.6.?-alpha. I've attached bug-related patches to this bug. I have created another issue for dependent enhancements. I have also created a bug for my remaining issue wi...I had several issues getting chutney working on OS X 10.9 with tor 0.2.6.?-alpha. I've attached bug-related patches to this bug. I have created another issue for dependent enhancements. I have also created a bug for my remaining issue with the chutney bridges* flavours accessing the global tor network, rather than the chutney network.
tor git d6b2a1709d28c656dadc019fb24145e6ac400771
chutney git 60b2d18d81904e6a71a352dfa8b5cb73f4e04003
= Getting chutney working
== 01-tor-TestingDirAuthVoteExit.patch
The authorities configured by chutney didn't apply the "Exit" flag to the exits in the network until about 35 minutes had passed. This patch creates a test config TestingDirAuthVoteExit which makes authorities mark particular relays as exits, similar to TestingDirAuthVoteGuard. (There may be a better way of getting the authorities to mark exits within seconds rather than half an hour.)
== 02-chutney-authority-force-vote-exit-guard.patch
Use tor-TestingDirAuthVoteExit.patch and TestingDirAuthVoteGuard to make all authorities vote all relays as exits and guards. This ensures that tor's make test-network will work. But it's a sledgehammer approach.
== 03-chutney-explicit-exit.patch
It looks like tor exits need to be explicitly configured to accept all ports on localhost (they won't do it by default).
== 04-chutney-basic-decreased-guards-more-relays.patch
Since the guard percentage has been reduced from 50% to 25% of relays, we need more relays to compensate.
= Fixing chutney issues
== 05-chutney-kill-old-nets-other-flavours.patch
If you change the chutney flavour between runs, tor processes from the previous run sometimes don't get terminated when the next chutney runs. This patch fixes this issue by using pidfiles, and, in the extreme case of kill-zombies.sh, ps and path grepping (which is a hack).
== 06-test-network-wait-25s.patch (optional)
The chutney network seems more reliable if I wait for 25 seconds instead of 18. YMMV
== 07-test-network-echo.patch
The default shell on OS X is bash, which has a builtin echo. When called in "sh" mode, this echo does not accept "-n". This patch uses "/bin/echo -n" instead.
= Debugging tor launched by chutney
== 08-chutney-enable-debugger.patch
This patch sets DisableDebuggerAttachment to 0, because if you're running a test network, you probably don't care about a debugger. (I may be wrong about this.)
== 09-chutney-TorNet-not-RunAsDaemon-comments.patch
I found it hard to debug tor processes launched by chutney when RunAsDaemon was set. So I wrote code that allowed the use of poll() instead of wait() to catch launch failures. This patch includes these changes in the comments - the functionality is the same as the git version of chutney. (There may be a better way of making this an option.)teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/13162[patch] Extra flavours for chutney and tor 0.2.6-alpha2020-06-27T13:19:06Zteor[patch] Extra flavours for chutney and tor 0.2.6-alphaAfter I got chutney working on OS X 10.9 with tor 0.2.6.?-alpha (see legacy/trac#13161), I created the attached basic+middle, large, and huge flavours.
tor git d6b2a1709d28c656dadc019fb24145e6ac400771 + legacy/trac#13161
chutney git 60b...After I got chutney working on OS X 10.9 with tor 0.2.6.?-alpha (see legacy/trac#13161), I created the attached basic+middle, large, and huge flavours.
tor git d6b2a1709d28c656dadc019fb24145e6ac400771 + legacy/trac#13161
chutney git 60b2d18d81904e6a71a352dfa8b5cb73f4e04003 + legacy/trac#13161
= Extra chutney configs
== 10-chutney-basic-middle.patch
Now that we have middle relays (relay.tmpl) and exits (exit.tmpl) from chutney-explicit-exit.patch in legacy/trac#13161, configure a network with middle & exit routers.
== 11-chutney-large-huge.patch
Add new large and huge chutney configs. Depends on chutney-explicit-exit.patch in legacy/trac#13161Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/13887Pick a reporting format for Chutney2020-07-23T20:07:43ZNick MathewsonPick a reporting format for ChutneyWe need some way for Chutney to tell another process about the events it's seeing.
Options include CTF, json, protocolbufs, line-based text.
See https://lists.torproject.org/pipermail/tor-dev/2014-December/007898.html for a synopsis of...We need some way for Chutney to tell another process about the events it's seeing.
Options include CTF, json, protocolbufs, line-based text.
See https://lists.torproject.org/pipermail/tor-dev/2014-December/007898.html for a synopsis of goals.
I'm especially interested in dgoulet's opinion on CTF in particular.
I'm especially interested in atagar's opinion about what would be friendliest to a stem-based chutney.https://gitlab.torproject.org/tpo/core/chutney/-/issues/13934Hidden service torrc template and network configuration2020-06-27T13:19:05ZDavid Gouletdgoulet@torproject.orgHidden service torrc template and network configurationAttached is a patch for HS support in chutney.
The size of the network is a bit bigger than usual because I needed a more nodes for performance measurements. Easy command to use it:
`chutney start networks/hs`
hs --> goes in networks/...Attached is a patch for HS support in chutney.
The size of the network is a bit bigger than usual because I needed a more nodes for performance measurements. Easy command to use it:
`chutney start networks/hs`
hs --> goes in networks/
hs.tmpl --> goes in torrc_templates/Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/13935Issue with HS availability2020-06-27T13:19:05ZDavid Gouletdgoulet@torproject.orgIssue with HS availabilityIt takes around 40 to 45 seconds for a client of the chutney network to be able to reach an hidden service.
My first guest would be to look at the upload period of the HS descriptor. I can see in the logs that the HSDir simply doesn't h...It takes around 40 to 45 seconds for a client of the chutney network to be able to reach an hidden service.
My first guest would be to look at the upload period of the HS descriptor. I can see in the logs that the HSDir simply doesn't have the descriptor when the client is trying to fetch it.
This can be reproduced by using the "patch" in legacy/trac#13934Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/13956Remove old configuration files before configuring2020-06-27T13:19:05ZTracRemove old configuration files before configuringIt is very simple but i think it is useful. Before creating a new configuration it removes the old configuration files. I have already made a pull request to https://github.com/torproject/chutney
**Trac**:
**Username**: huigIt is very simple but i think it is useful. Before creating a new configuration it removes the old configuration files. I have already made a pull request to https://github.com/torproject/chutney
**Trac**:
**Username**: huigNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/13980Added new script for warnings in chutney's logs2020-06-27T13:19:05ZTracAdded new script for warnings in chutney's logsIn chutney's repository in the TODO file there was a comment about doing a script for getting warnings in the logs. The script is at https://github.com/huig-/chutney and i made a pull request days ago (see https://trac.torproject.org/pro...In chutney's repository in the TODO file there was a comment about doing a script for getting warnings in the logs. The script is at https://github.com/huig-/chutney and i made a pull request days ago (see https://trac.torproject.org/projects/tor/ticket/13956#ticket) with a different change but now it also includes the script about warnings. The script basically gets warnings int net/nodes/*/info.logs and counts how many times each warning occurs. I have also added an option to get the warnings of only one node (specified by the user) instead of all of them.
The script is called warnings.sh
**Trac**:
**Username**: huigNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/13984Added new script for getting hs addresses in chutney2020-06-27T13:19:05ZTracAdded new script for getting hs addresses in chutneyThe script outputs the hidden services onion addresses in chutney. It is simple but I was told it would be useful. The script is at https://github.com/huig-/chutney and it is called hsaddress.sh (tools/hsaddress.sh).
**Trac**:
**Usern...The script outputs the hidden services onion addresses in chutney. It is simple but I was told it would be useful. The script is at https://github.com/huig-/chutney and it is called hsaddress.sh (tools/hsaddress.sh).
**Trac**:
**Username**: huigNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14019Fix Python 2 issues in chutney refactor2020-06-27T13:19:05ZteorFix Python 2 issues in chutney refactorPlease merge fix-python2-issues from https://github.com/teor2345/chutney.git
This fixes:
* Brackets around print statements
* Double-opening a filePlease merge fix-python2-issues from https://github.com/teor2345/chutney.git
This fixes:
* Brackets around print statements
* Double-opening a fileNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14067chutney verify doesn't verify bridge clients or hidden services2020-06-27T13:19:05Zteorchutney verify doesn't verify bridge clients or hidden servicesThe current `chutney verify` implementation only verifies clients.
I have a patch that makes it verify bridge clients as well. (I'll post it soon.)
But I don't know how to verify a hidden service - particularly the one configured in th...The current `chutney verify` implementation only verifies clients.
I have a patch that makes it verify bridge clients as well. (I'll post it soon.)
But I don't know how to verify a hidden service - particularly the one configured in the `hs` chutney network.
dgoulet, do you have any ideas?teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14173chutney verify: verify in parallel, rather than one at a time (Tor load / per...2020-06-27T13:19:04Zteorchutney verify: verify in parallel, rather than one at a time (Tor load / performance testing)There is a need to load-test Tor relays when bringing them up, and to performance-test tor in general, whether relaying or hidden services.
See:
Tor BSD Underperformance https://lists.torproject.org/pipermail/tor-dev/2015-January/008048...There is a need to load-test Tor relays when bringing them up, and to performance-test tor in general, whether relaying or hidden services.
See:
Tor BSD Underperformance https://lists.torproject.org/pipermail/tor-dev/2015-January/008048.html
Simulate high Tor load https://lists.torproject.org/pipermail/tor-relays/2015-January/006203.html
It would be nice if `chutney verify` connections were done in parallel, but I don't think that's the case.
Nick, is this SponsorR, or SponsorS, or both?Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14174chutney verify: make amount of data sent configurable (Tor load / performance...2020-06-27T13:19:04Zteorchutney verify: make amount of data sent configurable (Tor load / performance testing)There is a need to load-test Tor relays when bringing them up, and to performance-test tor in general, whether relaying or hidden services.
See:
Tor BSD Underperformance https://lists.torproject.org/pipermail/tor-dev/2015-January/008048...There is a need to load-test Tor relays when bringing them up, and to performance-test tor in general, whether relaying or hidden services.
See:
Tor BSD Underperformance https://lists.torproject.org/pipermail/tor-dev/2015-January/008048.html
Simulate high Tor load https://lists.torproject.org/pipermail/tor-relays/2015-January/006203.html
It's possible to manually modify the amount of random data sent while verifying connectivity (look for "Octets" in chutney's TorNet.py).
But it would be nice if this amount was configurable.
Nick, is this SponsorR, or SponsorS, or both?teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14175chutney verify: report client, bridge, and HS performance2020-06-27T13:19:04Zteorchutney verify: report client, bridge, and HS performanceThere is a need to load-test Tor relays when bringing them up, and to performance-test tor in general, whether relaying or hidden services.
See:
Tor BSD Underperformance https://lists.torproject.org/pipermail/tor-dev/2015-January/008048...There is a need to load-test Tor relays when bringing them up, and to performance-test tor in general, whether relaying or hidden services.
See:
Tor BSD Underperformance https://lists.torproject.org/pipermail/tor-dev/2015-January/008048.html
Simulate high Tor load https://lists.torproject.org/pipermail/tor-relays/2015-January/006203.html
It would be nice to have a command for this, that would execute the same tests, and then report performance as a series of MB/s figures for clients->exits, bridge clients->exits, and clients->hidden services.
Nick, is this SponsorR, or SponsorS, or both?teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14315Update README2020-06-27T13:19:04ZcypherpunksUpdate READMECurrently the README states that Python 2.5 or later is supported. Chutney already uses features that are present in Python 2.6 and later and Python 2.6 is not maintained.
Should Python 2.7 not be the minimum version?Currently the README states that Python 2.5 or later is supported. Chutney already uses features that are present in Python 2.6 and later and Python 2.6 is not maintained.
Should Python 2.7 not be the minimum version?Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14332Use new string formatting interface2020-07-23T20:09:01ZcypherpunksUse new string formatting interfaceThe Python documentation mentions issues with printf-style string formatting and recommends the new `str.format()` interface [0]. The new interface is used in some parts but not everywhere. The attached patch fixes this by using the new ...The Python documentation mentions issues with printf-style string formatting and recommends the new `str.format()` interface [0]. The new interface is used in some parts but not everywhere. The attached patch fixes this by using the new interface where printf-style is currently used.
Additionally, it solves several bugs in `lib/chutney/TorNet.py` caused by the printf-style separator (%) being outside of the `print()`. The bugs result in Chutney crashing when `tor` or `tor-gencert` is not in `PATH` and not specified through the environment variables.
[0] https://docs.python.org/3/library/stdtypes.html#printf-style-string-formattinghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14333Document the environment variables2020-06-27T13:19:04ZcypherpunksDocument the environment variablesThe environment variables used by Chutney are hidden in the code and not documented anywhere else. The attached patch mentions them in the `README`.The environment variables used by Chutney are hidden in the code and not documented anywhere else. The attached patch mentions them in the `README`.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/14954Chutney: add control socket to torrc templates2020-07-17T11:03:24ZDavid Gouletdgoulet@torproject.orgChutney: add control socket to torrc templatesHaving a control socket is *very* useful in a test network especially when you want to test control events! :)
It should be added to the common.i file in torrc_templates/ so every node has one by default.Having a control socket is *very* useful in a test network especially when you want to test control events! :)
It should be added to the common.i file in torrc_templates/ so every node has one by default.https://gitlab.torproject.org/tpo/core/chutney/-/issues/15229Design document for chutney22020-07-23T20:09:24ZNick MathewsonDesign document for chutney2I should write up my ideas for exapanding and refactoring chutney to be more flexible and powerful.I should write up my ideas for exapanding and refactoring chutney to be more flexible and powerful.https://gitlab.torproject.org/tpo/core/chutney/-/issues/15231Prototype implementation for the main ideas for chutney22020-07-23T20:09:29ZNick MathewsonPrototype implementation for the main ideas for chutney2We should have it running at least a small handful of network tests in a reasonable way.We should have it running at least a small handful of network tests in a reasonable way.https://gitlab.torproject.org/tpo/core/chutney/-/issues/15353Some chutney tests fail when localhost is the only available IP2020-06-27T13:19:03ZteorSome chutney tests fail when localhost is the only available IPWhen I turn off the Wifi / Internet connection on my Mac (OS X 10.10.2), the chutney Exit tests fail with 'no exit to that port', but the hidden service tests are fine.
This is probably a misconfiguration around the localhost exit ports...When I turn off the Wifi / Internet connection on my Mac (OS X 10.10.2), the chutney Exit tests fail with 'no exit to that port', but the hidden service tests are fine.
This is probably a misconfiguration around the localhost exit ports (all ports should be allowed, or, at the very least, the ports chutney uses should be allowed).
I'll see if I can find time to track this down and fix it, but it's low priority, as most people run chutney on a computer with an IP address apart from localhost.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/15418chutney: add debug command-line flag2020-06-27T13:19:03Zteorchutney: add debug command-line flagchutney has a debug variable `debug_flag` in `Traffic.py`.
This would be much easier to use if it was set from the command-line, or even automatically activated (at some level) on transmission failure.chutney has a debug variable `debug_flag` in `Traffic.py`.
This would be much easier to use if it was set from the command-line, or even automatically activated (at some level) on transmission failure.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/15419chutney: on failure, explain which connection failed2020-07-23T21:08:15Zteorchutney: on failure, explain which connection failedWhen there's a failure in the chutney data transmission tests, it would be really, really helpful to know which connection failed. The debug flag doesn't quite do this yet, but it's pretty close.
This is particularly important now that ...When there's a failure in the chutney data transmission tests, it would be really, really helpful to know which connection failed. The debug flag doesn't quite do this yet, but it's pretty close.
This is particularly important now that chutney supports three different types of connectivity tests (in the same network):
* client -> exit
* bridge client -> exit
* [bridge] client -> hidden service
Over two different protocols:
* IPv4
* IPv6 (currently bridges only)https://gitlab.torproject.org/tpo/core/chutney/-/issues/15430include a net directory2020-06-27T13:19:03ZSebastian Hahninclude a net directoryWhen launching tests from inside Tor, settings a CHUTNEY_PATH to a clean git checkout doesn't work because chutney doesn't include a net/ directory in its repo. Added one in branch net_dir in my chutney repoWhen launching tests from inside Tor, settings a CHUTNEY_PATH to a clean git checkout doesn't work because chutney doesn't include a net/ directory in its repo. Added one in branch net_dir in my chutney repoNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/15783On chutney verify failure, inform user about debug mode2020-06-27T13:19:03ZteorOn chutney verify failure, inform user about debug modeWhen chutney verify fails, tell the user how to activate debug mode by editing the source code.
Eventually, we'll make debug mode an argument, see legacy/trac#15418.When chutney verify fails, tell the user how to activate debug mode by editing the source code.
Eventually, we'll make debug mode an argument, see legacy/trac#15418.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/15784Clarify Chutney exit policies for IPv4 and IPv6, including private addresses2020-06-27T13:19:03ZteorClarify Chutney exit policies for IPv4 and IPv6, including private addressesAllow chutney exits to exit to all private addresses.
Tidy up exit policies, so each part of the policy is clearly identified.
Document alternatives to work around legacy/trac#11264, the microdescriptor 2x /8 requirement for exits.
Pr...Allow chutney exits to exit to all private addresses.
Tidy up exit policies, so each part of the policy is clearly identified.
Document alternatives to work around legacy/trac#11264, the microdescriptor 2x /8 requirement for exits.
Prepare for resolving legacy/trac#15353 by making sure the exit policies *should* work when localhost is the only available IP. This still requires further investigation.
Patch:
**Branch:** clarify-exit-policies
**Repository:** https://github.com/teor2345/chutney.gitteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/15860chutney verify: make number of connections configurable2020-06-27T13:19:02Zteorchutney verify: make number of connections configurableIn order to replicate legacy/trac#15463 and similar, we need to make lots of client connections to the same HS.
chutney can easily be modified to do this.In order to replicate legacy/trac#15463 and similar, we need to make lots of client connections to the same HS.
chutney can easily be modified to do this.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/15936Add additional HS networks to chutney2020-06-27T13:19:02ZteorAdd additional HS networks to chutneylegacy/trac#8902 needs some larger HS networks for testing multiple client connections.
This ticket reminds me to create a branch to share them with dgoulet and others.
Tags copied from legacy/trac#8902.legacy/trac#8902 needs some larger HS networks for testing multiple client connections.
This ticket reminds me to create a branch to share them with dgoulet and others.
Tags copied from legacy/trac#8902.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16310chutney thinks tor fails to launch under high load2020-06-27T13:19:02Zteorchutney thinks tor fails to launch under high loadUnder high CPU load, chutney launches tor processes successfully, but thinks that some of the processes have failed to launch.
Unless this is a particular issue for anyone, I suspect the answer will be "don't do that", and to fix it in ...Under high CPU load, chutney launches tor processes successfully, but thinks that some of the processes have failed to launch.
Unless this is a particular issue for anyone, I suspect the answer will be "don't do that", and to fix it in chutney 2.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16386Chutney generates network with no bandwidth weights2020-07-23T21:07:50ZGeorge KadianakisChutney generates network with no bandwidth weightsHello,
when chutney makes a Tor network, it sets the following options in the dirauth torrc:
```
TestingDirAuthVoteExit *
TestingDirAuthVoteGuard *
```
which results in a network where all nodes are guards and exits.
This results in to...Hello,
when chutney makes a Tor network, it sets the following options in the dirauth torrc:
```
TestingDirAuthVoteExit *
TestingDirAuthVoteGuard *
```
which results in a network where all nodes are guards and exits.
This results in totally unbalanced bandwidths (0 guard bandwidth, 0 middle bandwidth, full guard+exit bandwidth) and authorities will not generate bandwidth weights:
```
[warn] Consensus with empty bandwidth: G=0 M=0 E=0 D=65770 T=65770
```
This is quite different from the real Tor network.
It would be great if chutney could assign Exit/Guard flag in a smarter way to only a few nodes, so that bandwidth can be allocated more widely.https://gitlab.torproject.org/tpo/core/chutney/-/issues/16438Enable control port in chutney Tor clients2020-06-27T13:19:02ZdonnchaEnable control port in chutney Tor clientsI'd like to be able to use chutney to perform integration testing for OnionBalance. I need access to a Tor client's control port but the control port is not enabled in the current client.tmpl.
I've pushed a trivial patch to my Github re...I'd like to be able to use chutney to perform integration testing for OnionBalance. I need access to a Tor client's control port but the control port is not enabled in the current client.tmpl.
I've pushed a trivial patch to my Github repo at https://github.com/DonnchaC/chutney/pull/1/filesNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16454chutney's "tor does not support this option" is unclear2020-06-27T13:19:02Zteorchutney's "tor does not support this option" is unclearA user emailed me concerned that chutney was giving a lot of "tor does not support this option" errors.
These messages could be much clearer:
* The message is a warning, not an error
* The message doesn't describe the likely consequence...A user emailed me concerned that chutney was giving a lot of "tor does not support this option" errors.
These messages could be much clearer:
* The message is a warning, not an error
* The message doesn't describe the likely consequences. We could document these, at least for the 3 `TestingDirAuthVote*` options
I also wrote back to them:
```
This is a warning that happens when you have an older version of tor. It is telling you that your version of tor doesn't support one of the options used in the standard chutney torrc templates. Chutney warns you, but disables the option so that tor will still run.
This particular option is used to speed up the authorities' votes for hidden service directories. If you are running a chutney network with a hidden service, you can speed up bootstrap by using tor 0.2.6.9 or tor 0.2.7-alpha.
If you're not using hidden services, you can disregard this warning.
Thank you for letting me know, I will log a task to update the documentation to make it clearer that this is a warning, and explain the likely consequence.
```https://gitlab.torproject.org/tpo/core/chutney/-/issues/16802Enable sandbox in chutney2020-06-27T13:19:02ZNick MathewsonEnable sandbox in chutneyIt would be good if we tried to run Chutney (whenever possible) with the sandbox turned on, so we can get early notice when we break simple things like "run a relay with the sandbox turned on."It would be good if we tried to run Chutney (whenever possible) with the sandbox turned on, so we can get early notice when we break simple things like "run a relay with the sandbox turned on."Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16804Add more chutney networks to "make test-network" or such2020-06-27T13:19:02ZNick MathewsonAdd more chutney networks to "make test-network" or suchThe tor 'make test-network' script runs some chutney tests and makes sure they work.
We could get improved integration testing by including hidden services and pluggable transports and bridges, and IPv6 and mixed networks, etc etc, for ...The tor 'make test-network' script runs some chutney tests and makes sure they work.
We could get improved integration testing by including hidden services and pluggable transports and bridges, and IPv6 and mixed networks, etc etc, for instance. Some of these are available now, some would require chutney extensions.Tor: 0.2.7.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16902chutney fails to find python2 on stock OS X install2020-06-27T13:19:01Zteorchutney fails to find python2 on stock OS X installOS X Yosemite's /usr/bin/python is python 2.7.10, but OS X doesn't provide a python2 symlink (which chutney expects).
This causes chutney to fail out of the box on OS X.OS X Yosemite's /usr/bin/python is python 2.7.10, but OS X doesn't provide a python2 symlink (which chutney expects).
This causes chutney to fail out of the box on OS X.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16904chutney fails to work with python32020-06-27T13:19:01Zteorchutney fails to work with python3I thought we had this working at one stage, but maybe my recent changes broke it.I thought we had this working at one stage, but maybe my recent changes broke it.cypherpunkscypherpunkshttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16946Create new chutney bridges+hs network2020-06-27T13:19:01ZteorCreate new chutney bridges+hs networkAdd bridges+hs network to test bridges and hidden services together.
This network allows non-IPv6 systems to test almost all tor functionality.Add bridges+hs network to test bridges and hidden services together.
This network allows non-IPv6 systems to test almost all tor functionality.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/16949Make Chutney Easier to Use & More Functional, Write Guide2020-07-23T20:13:24ZteorMake Chutney Easier to Use & More Functional, Write GuideI (teor) am writing a guide to using chuntney.
As part of that, I'm making chuntey easier to use, mainly through a series of tweaks that simplify usage and improve test coverage.
See https://trac.torproject.org/projects/tor/wiki/doc/Tor...I (teor) am writing a guide to using chuntney.
As part of that, I'm making chuntey easier to use, mainly through a series of tweaks that simplify usage and improve test coverage.
See https://trac.torproject.org/projects/tor/wiki/doc/TorChutneyGuide and the child tickets for more details.https://gitlab.torproject.org/tpo/core/chutney/-/issues/16950Add a chutney command that checks if network bootstrap has finished2021-03-05T13:31:18ZteorAdd a chutney command that checks if network bootstrap has finishedThis command ideally needs to check (or wait before returning until) all clients are ready to send to all endpoints:
* Authorities: consensus is working
* Other Nodes have a copy of the consensus
* Hidden Services have uploaded descripto...This command ideally needs to check (or wait before returning until) all clients are ready to send to all endpoints:
* Authorities: consensus is working
* Other Nodes have a copy of the consensus
* Hidden Services have uploaded descriptors
* Anything else?https://gitlab.torproject.org/tpo/core/chutney/-/issues/16951Reduce frequency of intermittent chutney failures2020-06-27T13:19:01ZteorReduce frequency of intermittent chutney failuresWhen run repeatedly in a loop, chutney fails to launch or bootstrap every so often. Why is this, and can we fix it?When run repeatedly in a loop, chutney fails to launch or bootstrap every so often. Why is this, and can we fix it?https://gitlab.torproject.org/tpo/core/chutney/-/issues/16954Refactor chutney networks: increase configurability, reduce code duplication2020-07-23T20:14:00ZteorRefactor chutney networks: increase configurability, reduce code duplicationRefactor the existing chutney networks to increase configurability and reduce code duplication.
Make it easier to mix and match particular authority, middle relay, client (standard, bridge, IPv6 bridge) and endpoint (exit, hidden service...Refactor the existing chutney networks to increase configurability and reduce code duplication.
Make it easier to mix and match particular authority, middle relay, client (standard, bridge, IPv6 bridge) and endpoint (exit, hidden service) nodes.
Also make version mixing easier.
Have a replaceable minimal default?
3 authorities, 1 client, 1 bridge client, 1 bridge, 1 middle, 1 exit, 1 hs
* each network includes 3 authorities, 1 client or bridge client, 1 exit or hs, and however many middles are required to do a fast bootstrap (assuming reachability)
* work out a way to select between client / bridge client, and exit or hs endpoints
* create a way to do a full bootstrap (no assuming reachability) by flipping one setting (change torrc options, add middle relays to some minimum of auth + exit + middle: 4? 5?)
* work out a way to add IPv6 bridges, old tor versions, and other features I'm forgetting
* work out a way to scale network size by multiplication, either of all nodes, or of particular node roles (start point, relay, end point), or of individual node configurations
* work out what syntax Python needs to include one configuration in another
Maybe it could look like this:
* core: authorities, non-exit relays?
* client: client, bridge, IPv6 bridge (choose at least one)
* endpoint: exit, hs (choose at least one)
* bootstrap: fast or full (choose one)
* version: built, system (choose at least one)
Then combined to give us all the networks that currently exist, without needing to cut and paste between network configurations.https://gitlab.torproject.org/tpo/core/chutney/-/issues/16999chutney torrcs contain duplicate TestingDirAuthVoteExit entries2020-06-27T13:19:00Zteorchutney torrcs contain duplicate TestingDirAuthVoteExit entriesasn reports that chutney-generated torrc files contain multiple TestingDirAuthVoteExit lines.
This is confusing and makes chutney hard to use. We should only have one entry for TestingDirAuthVote{Guard,Exit,HSDir} in each chutney-genera...asn reports that chutney-generated torrc files contain multiple TestingDirAuthVoteExit lines.
This is confusing and makes chutney hard to use. We should only have one entry for TestingDirAuthVote{Guard,Exit,HSDir} in each chutney-generated torrc file.https://gitlab.torproject.org/tpo/core/chutney/-/issues/17002chutney start could tell users how to kill old chutney tors2020-06-27T13:19:00Zteorchutney start could tell users how to kill old chutney torsThe most common reason chutney start fails to start all tor processes is port conflicts caused by old chutney tor processes still running.
We could tell users how to kill processes (should we?) using a summary of the following informati...The most common reason chutney start fails to start all tor processes is port conflicts caused by old chutney tor processes still running.
We could tell users how to kill processes (should we?) using a summary of the following information from https://trac.torproject.org/projects/tor/wiki/doc/TorChutneyGuide
"However, sometimes chutney fails to launch due to a persistent error. The most common issue is having a previous chutney network active. Old chutney networks are typically killed by chutney/tools/bootstrap-network.sh using chutney stop, but sometimes this doesn't happen. The tor processes launched by chutney can be found using a command like `ps auxwww | grep chutney | grep tor`. Ensure each pid doesn't belong to a process you want, like a system tor or an unrelated process, then kill each chutney tor pid using `kill`. Alternately, each tor instance's pid files can be found in the chutney/net/nodes directory. (Old nodes directories are renamed using an increasing numeric timestamp as a suffix.)"
Use:
* chutney stop
* `ps auxwww | grep chutney | grep tor` and carefull check each pid before using `kill` on it
* the tor pid files in the chutney/net/nodes* directories (maybe, these can be very old)https://gitlab.torproject.org/tpo/core/chutney/-/issues/17013Does chutney need to test various rare IPv6/IPv4 combinations?2020-07-23T20:15:32ZteorDoes chutney need to test various rare IPv6/IPv4 combinations?We could make sure that legacy/trac#16954 enables the following combinations:
* IPv6 bridge client - IPv6 bridge -- ---- hs
* IPv6 bridge client - IPv6 bridge -- IPv6-only Exit
* IPv6 bridge client - IPv6 bridge -- IPv6/IPv4 Exit
* IPv6...We could make sure that legacy/trac#16954 enables the following combinations:
* IPv6 bridge client - IPv6 bridge -- ---- hs
* IPv6 bridge client - IPv6 bridge -- IPv6-only Exit
* IPv6 bridge client - IPv6 bridge -- IPv6/IPv4 Exit
* IPv6 bridge client - IPv6 bridge -- IPv4 Exit
* IPv4 bridge client - IPv4 bridge -- IPv6-only Exit
* IPv4 bridge client - IPv4 bridge -- IPv6/IPv4 Exit
This probably also depends on legacy/trac#17011.
* IPv4 client --- IPv6-only Exit
* IPv4 client --- IPv6/IPv4 Exithttps://gitlab.torproject.org/tpo/core/chutney/-/issues/17014chutney: automate discovery and use of different tor versions2020-07-23T20:16:05Zteorchutney: automate discovery and use of different tor versionsBuilding on legacy/trac#17015, we could:
* Preserve the old path before test-network.sh modifies it, and use the first tor in that path (legacy/trac#17015)
* Check for /usr/bin/tor and /usr/local/bin/tor (And maybe /opt ?) (legacy/trac#1...Building on legacy/trac#17015, we could:
* Preserve the old path before test-network.sh modifies it, and use the first tor in that path (legacy/trac#17015)
* Check for /usr/bin/tor and /usr/local/bin/tor (And maybe /opt ?) (legacy/trac#17015)
* Check the version of each tor binary, find a minimal set of different tor versions, then use each version in a round-robin fashionhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/17015chutney should try harder to find a stable/different version of tor2020-07-23T20:15:49Zteorchutney should try harder to find a stable/different version of torCurrently, chutney tries to find a stable version of tor under tor-stable anywhere in the path, then stops.
We could:
* Preserve the old path before test-network.sh modifies it, and use the first tor in that path
* Check for /usr/bin/to...Currently, chutney tries to find a stable version of tor under tor-stable anywhere in the path, then stops.
We could:
* Preserve the old path before test-network.sh modifies it, and use the first tor in that path
* Check for /usr/bin/tor and /usr/local/bin/tor (And maybe /opt ?)
* use each version in a round-robin fashion (Split to legacy/trac#17014, child of legacy/trac#16954)https://gitlab.torproject.org/tpo/core/chutney/-/issues/17088chutney shouldn't set DirPort on bridge relays2020-06-27T13:19:00Zteorchutney shouldn't set DirPort on bridge relays```
../chutney/net/nodes/008br/notice.log:Sep 15 23:24:25.859 [warn] Can't set a DirPort on a bridge relay; disabling DirPort
``````
../chutney/net/nodes/008br/notice.log:Sep 15 23:24:25.859 [warn] Can't set a DirPort on a bridge relay; disabling DirPort
```https://gitlab.torproject.org/tpo/core/chutney/-/issues/17089Consider turning off geoip for chutney bridges2020-06-27T13:18:59ZteorConsider turning off geoip for chutney bridges```
../chutney/net/nodes/008br/notice.log:Sep 15 23:24:25.000 [warn] Failed to open GEOIP file /usr/local/share/tor/geoip. We've been configured to see which countries can access us as a bridge, and we need GEOIP information to tell whi...```
../chutney/net/nodes/008br/notice.log:Sep 15 23:24:25.000 [warn] Failed to open GEOIP file /usr/local/share/tor/geoip. We've been configured to see which countries can access us as a bridge, and we need GEOIP information to tell which countries clients are in.
```https://gitlab.torproject.org/tpo/core/chutney/-/issues/17090chutney triggers tor warnings about ExitRelay not being set2020-06-27T13:18:59Zteorchutney triggers tor warnings about ExitRelay not being setAll authorities including the bridge authority, all relays:
```
../chutney/net/nodes/000a/info.log:Sep 15 23:24:25.144 [warn] Tor is running as an exit relay with the default exit policy. If you did not want this behavior, please set the...All authorities including the bridge authority, all relays:
```
../chutney/net/nodes/000a/info.log:Sep 15 23:24:25.144 [warn] Tor is running as an exit relay with the default exit policy. If you did not want this behavior, please set the ExitRelay option to 0. If you do want to run an exit Relay, please set the ExitRelay option to 1 to disable this warning, and for forward compatibility.
../chutney/net/nodes/000a/info.log:Sep 15 23:24:25.144 [warn] In a future version of Tor, ExitRelay 0 may become the default when no ExitPolicy is given.
```
This affects legacy/trac#17011 when we are trying to force IPv6-only exit connections.
This may also indicate a misconfiguration, or simply me (teor) forgetting `ExitRelay 0/1` in the torrc templates.https://gitlab.torproject.org/tpo/core/chutney/-/issues/17276Make at least one authority script integrated with Chutney2020-07-23T20:16:27ZNick MathewsonMake at least one authority script integrated with Chutneyhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/17277Chutney tests should include misbehaving clients and servers2020-06-27T13:18:59ZNick MathewsonChutney tests should include misbehaving clients and serversNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/17290Include network tests with ill-behaved clients and servers2020-07-23T20:17:17ZNick MathewsonInclude network tests with ill-behaved clients and serversThis is a deliverable for October 2016.This is a deliverable for October 2016.https://gitlab.torproject.org/tpo/core/chutney/-/issues/17622chutney rendezvous single onion services templates & network2020-06-27T13:18:59Zteorchutney rendezvous single onion services templates & network~~Please merge the feature-17178-rsos branch in https://github.com/teor2345/chutney.git~~
I'll make this happen when legacy/trac#17178 closes.~~Please merge the feature-17178-rsos branch in https://github.com/teor2345/chutney.git~~
I'll make this happen when legacy/trac#17178 closes.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/17629Create chutney templates that match public tor network timings2020-07-23T20:22:10ZteorCreate chutney templates that match public tor network timingshttps://gitlab.torproject.org/tpo/core/chutney/-/issues/17631chutney fails on systems with python2, but no python2020-06-27T13:18:58Zteorchutney fails on systems with python2, but no pythonI forgot brackets when I rewrite the command checks.I forgot brackets when I rewrite the command checks.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/17812Chutney tests for IPv6-only clients2020-06-27T13:18:58ZteorChutney tests for IPv6-only clientsOnce clients can bootstrap over IPv6 (legacy/trac#17281), chutney should add a torrc template and network containing IPv6-only clientsOnce clients can bootstrap over IPv6 (legacy/trac#17281), chutney should add a torrc template and network containing IPv6-only clientshttps://gitlab.torproject.org/tpo/core/chutney/-/issues/18185Fix coding style according to PEP82020-06-27T13:18:58ZcypherpunksFix coding style according to PEP8The pep8 tool shows several minor PEP8 violations. Fixing these violations would makes it easier for future developers (including me) to check whether their patches are according to the PEP8 without having to ignore existing violations. ...The pep8 tool shows several minor PEP8 violations. Fixing these violations would makes it easier for future developers (including me) to check whether their patches are according to the PEP8 without having to ignore existing violations. Also having existing violations gives future developers less incentive to fix violations they introduce.cypherpunkscypherpunkshttps://gitlab.torproject.org/tpo/core/chutney/-/issues/18341Chutney won't run because some print() statements are borken2020-06-27T13:18:58ZIsis LovecruftChutney won't run because some print() statements are borken```
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pk...```
The above is a description of an error in a Python program. Here is
the original traceback:
Traceback (most recent call last):
File "/usr/lib/python2.7/runpy.py", line 162, in _run_module_as_main
"__main__", fname, loader, pkg_name)
File "/usr/lib/python2.7/runpy.py", line 72, in _run_code
exec code in run_globals
File "/home/isis/code/torproject/chutney/lib/chutney/TorNet.py", line 1180, in <module>
sys.exit(main())
File "/home/isis/code/torproject/chutney/lib/chutney/TorNet.py", line 1174, in main
result = runConfigFile(args['action'], f)
File "/home/isis/code/torproject/chutney/lib/chutney/TorNet.py", line 1151, in runConfigFile
return getattr(network, verb)()
File "/home/isis/code/torproject/chutney/lib/chutney/TorNet.py", line 841, in configure
b.preConfig(network)
File "/home/isis/code/torproject/chutney/lib/chutney/TorNet.py", line 320, in preConfig
self._genAuthorityKey()
File "/home/isis/code/torproject/chutney/lib/chutney/TorNet.py", line 387, in _genAuthorityKey
"path, or put the binary into $PATH.") % tor_gencert
TypeError: unsupported operand type(s) for %: 'NoneType' and 'str'
```
Umm… the `%` operator goes _inside_ the `print()` function call. Otherwise, you're attempting to do a format string operation on the return value of the `print()` function (which is `None`).Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/18482Add fallback directory test schedules to chutney2020-06-27T13:18:58ZteorAdd fallback directory test schedules to chutneyIn legacy/trac#18481, I remove the test schedules from legacy/trac#4483 from tor.
I want to add them to the default template in chutney.In legacy/trac#18481, I remove the test schedules from legacy/trac#4483 from tor.
I want to add them to the default template in chutney.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/18822Add fallback dir schedule to chutney common template2020-07-23T20:19:02ZDavid Gouletdgoulet@torproject.orgAdd fallback dir schedule to chutney common templateThis is for part of legacy/trac#18481. We should put testing values in the chutney common template.This is for part of legacy/trac#18481. We should put testing values in the chutney common template.https://gitlab.torproject.org/tpo/core/chutney/-/issues/18826Allow setting the listen address for all Chutney hosts2020-06-27T13:18:57ZanonymAllow setting the listen address for all Chutney hostsIn Tails' automated test suite we have plans to use Chutney to run a simulated Tor network (for increased performance and, in particular, robustness). Of course, we do not want to run that Chutney instance in the system under testing (i....In Tails' automated test suite we have plans to use Chutney to run a simulated Tor network (for increased performance and, in particular, robustness). Of course, we do not want to run that Chutney instance in the system under testing (i.e. the Tails instance) but on the host orchestrating the tests. In order to avoid having to forward traffic via firewall rules, we'd like it to be possible to make the Chutney hosts listen on some other IP address than localhost.
The attached patch allows setting the address all Chutney hosts will listen on via the `CHUTNEY_LISTEN_ADDRESS` environment variable.anonymanonymhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/18827Allow specifying the Chutney data directory2020-06-27T13:18:57ZanonymAllow specifying the Chutney data directoryWith "Chutney data directory" I am referring to the `net` sub-directory. I find it preferable to be able to store such cruft outside of the source tree. See the attached patch.With "Chutney data directory" I am referring to the `net` sub-directory. I find it preferable to be able to store such cruft outside of the source tree. See the attached patch.anonymanonymhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/18882chutney reads the system torrc when generating the router key2020-06-27T13:18:57Zanonymchutney reads the system torrc when generating the router keyThis can easily be demonstrated by adding an invalid option to `/etc/tor/torrc` and then running `chutney configure`. This feels wrong, and will fail if you run a hidden service on the system-wide Tor instance.This can easily be demonstrated by adding an invalid option to `/etc/tor/torrc` and then running `chutney configure`. This feels wrong, and will fail if you run a hidden service on the system-wide Tor instance.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/18932Re-launching chutney with cached descriptors sometimes fails2020-07-23T20:18:37ZteorRe-launching chutney with cached descriptors sometimes failsA user reports that chutney sometimes fails when relaunched with a previous configuration and cached descriptors.
https://lists.torproject.org/pipermail/tor-dev/2016-April/010854.html
Typically, chutney is run on a newly-configured dir...A user reports that chutney sometimes fails when relaunched with a previous configuration and cached descriptors.
https://lists.torproject.org/pipermail/tor-dev/2016-April/010854.html
Typically, chutney is run on a newly-configured directory every time, so we don't run into this bug very often.https://gitlab.torproject.org/tpo/core/chutney/-/issues/19116Add scripts to chutney so it works on a tor binary2020-06-27T13:18:57ZteorAdd scripts to chutney so it works on a tor binaryIf chutney was self-contained and could run tests on a plain tor binary, it would be easier to do jenkins tests using chutney.
To make chutney work, we use parts of the tor git tree, including some Makefile targets to select the tests w...If chutney was self-contained and could run tests on a plain tor binary, it would be easier to do jenkins tests using chutney.
To make chutney work, we use parts of the tor git tree, including some Makefile targets to select the tests we run, and src/test/test-network.sh.
If we move these scripts to the chutney tree, we can run a set of chutney targets anywhere, without requiring a tor build tree.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/19533Document CHUTNEY_LISTEN_ADDRESS2020-06-27T13:18:57ZcypherpunksDocument CHUTNEY_LISTEN_ADDRESSThe environment variable was added in legacy/trac#18826 but was not added to the README file.The environment variable was added in legacy/trac#18826 but was not added to the README file.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/19545Ignore vim swap files and git mergetool backups2020-06-27T13:18:56ZcypherpunksIgnore vim swap files and git mergetool backupsWhile working on legacy/trac#9087 i noticed the Chutney git ignore list does not ignore vim swap files and git mergetool backup files.
The attached patches add both file extensions to the git ignore list.While working on legacy/trac#9087 i noticed the Chutney git ignore list does not ignore vim swap files and git mergetool backup files.
The attached patches add both file extensions to the git ignore list.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/19694Add a leekspin mode to chutney2020-07-23T20:23:33ZteorAdd a leekspin mode to chutneyLeekspin is a descriptor generator/fuzzer.
I'd like to be able to do the following:
* launch a test network using chutney
* feed the authorities (or HSDirs) descriptors generated by leekspin, just like a normal relay (hidden service) wo...Leekspin is a descriptor generator/fuzzer.
I'd like to be able to do the following:
* launch a test network using chutney
* feed the authorities (or HSDirs) descriptors generated by leekspin, just like a normal relay (hidden service) would
* watch what happens
I'd also like to be able to:
* dump the relay (hidden service) descriptors at the end of each chutney run
* feed the authorities descriptors generated in a previous chutney run (whether generated via leekspin or not)https://gitlab.torproject.org/tpo/core/chutney/-/issues/19701Allow tor and tor-gencert to be specified on the chutney command-line2020-06-27T13:18:56ZteorAllow tor and tor-gencert to be specified on the chutney command-lineAs well as being specified via the CHUTNEY_TOR and CHUTNEY_TOR_GENCERT environmental variables.As well as being specified via the CHUTNEY_TOR and CHUTNEY_TOR_GENCERT environmental variables.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/19764Add a "stop time" feature to chutney2020-06-27T13:18:56ZteorAdd a "stop time" feature to chutneyCurrently, chutney's test-network.sh waits for the following amount of time:
* a mandatory 15 seconds for the consensus to form
* up to 60 seconds while verifying the network
And then it terminates the network.
But sometimes I want chut...Currently, chutney's test-network.sh waits for the following amount of time:
* a mandatory 15 seconds for the consensus to form
* up to 60 seconds while verifying the network
And then it terminates the network.
But sometimes I want chutney to hang around for several minutes, or even forever.
There should be an option / environmental variable for this "stop time" in test-network.sh.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/19765Add a control port to every chutney tor instance, not just clients2020-06-27T13:18:56ZteorAdd a control port to every chutney tor instance, not just clientsSometimes it's handy to have a control port on a relay.Sometimes it's handy to have a control port on a relay.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/20036Chutney should allow single onion and tor2web modes on any HS / client2020-07-23T20:23:51ZteorChutney should allow single onion and tor2web modes on any HS / clientAt the moment, chutney users have to specifically configure single onion mode and tor2web mode.
It would be nice to allow any chutney net to be run with its hidden services in single onion mode, perhaps using config options that are con...At the moment, chutney users have to specifically configure single onion mode and tor2web mode.
It would be nice to allow any chutney net to be run with its hidden services in single onion mode, perhaps using config options that are conditional on environmental variables.
This would be harder with tor2web, because we'd need a separate tor binary as well. And it might not be worth the effort if tor2web isn't part of next-generation hidden services.https://gitlab.torproject.org/tpo/core/chutney/-/issues/20091Add a required version setting to chutney (or to chutney networks)2020-07-23T21:16:39ZteorAdd a required version setting to chutney (or to chutney networks)This is important when we add a new chutney feature or network (like in legacy/trac#20069), then make Tor depend on it.
One way we could do this is to check the exit status of chutney in chutney/chutney, and logging the appropriate mess...This is important when we add a new chutney feature or network (like in legacy/trac#20069), then make Tor depend on it.
One way we could do this is to check the exit status of chutney in chutney/chutney, and logging the appropriate message if it fails.
Another way is for the Tor chutney tests to set CHUTNEY_MIN_VERSION, and then chutney warns if it's not that version. This would actually require us to version chutney.https://gitlab.torproject.org/tpo/core/chutney/-/issues/20344Do chutney releases with semantic versioning2020-07-23T20:26:33ZteorDo chutney releases with semantic versioninghttps://gitlab.torproject.org/tpo/core/chutney/-/issues/20409When some chutney tors die, clean up the rest2020-06-27T13:18:54ZteorWhen some chutney tors die, clean up the restChutney should clean up all the tor processes it starts, and leave all other processes alone.
I think this means adding `pkill -P $$` to the end of chutney/tools/test-network.sh , and making sure it's executed even if chutney returns fa...Chutney should clean up all the tor processes it starts, and leave all other processes alone.
I think this means adding `pkill -P $$` to the end of chutney/tools/test-network.sh , and making sure it's executed even if chutney returns failure.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/20473Fix Chutney Nodes that don't bootstrap2020-06-27T13:18:54ZteorFix Chutney Nodes that don't bootstrapIn https://trac.torproject.org/projects/tor/ticket/14881#comment:58 , pastly reports:
> {{{
> ./nodes.basic-min/000a/ probably failed to bootstrap
> ./nodes.basic-min/001a/ probably failed to bootstrap
> ./nodes.basic-min/002r/ probably ...In https://trac.torproject.org/projects/tor/ticket/14881#comment:58 , pastly reports:
> {{{
> ./nodes.basic-min/000a/ probably failed to bootstrap
> ./nodes.basic-min/001a/ probably failed to bootstrap
> ./nodes.basic-min/002r/ probably failed to bootstrap
> ./nodes.bridges-ipv6-min/000a/ probably failed to bootstrap
> ./nodes.bridges-ipv6-min/001ba/ probably failed to bootstrap
> ./nodes.bridges-ipv6-min/002r/ probably failed to bootstrap
> ./nodes.bridges-min/000a/ probably failed to bootstrap
> ./nodes.bridges-min/001ba/ probably failed to bootstrap
> ./nodes.bridges-min/002r/ probably failed to bootstrap
> ./nodes.ipv6-exit-min/000a/ probably failed to bootstrap
> ./nodes.ipv6-exit-min/001a/ probably failed to bootstrap
> ./nodes.ipv6-exit-min/002r/ probably failed to bootstrap
> }}}
We should fix this.
We should also integrate this check into chutney.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/20513Use public tor network directory authority options in chutney2020-07-23T20:27:24ZteorUse public tor network directory authority options in chutneyChutney currently runs authorities with most options at their defaults.
But the directory authorities on the public tor network run with a set of standard options.
It would be great to use those options as much as possible in chutney (o...Chutney currently runs authorities with most options at their defaults.
But the directory authorities on the public tor network run with a set of standard options.
It would be great to use those options as much as possible in chutney (or even run most networks with them on, and one with them off), so that we are actually testing the code that is running on the real authorities.
For example, when legacy/trac#18319 is implemented, the directory authorities will set AuthDirPinKeys 1 (the default is 0).https://gitlab.torproject.org/tpo/core/chutney/-/issues/20547Fix typo in Chutney's bootstrap-network.sh2020-06-27T13:18:54ZTracFix typo in Chutney's bootstrap-network.shThere's a one-character typo in bootstrap-network.sh. The attached patch fixes it.
**Trac**:
**Username**: larslThere's a one-character typo in bootstrap-network.sh. The attached patch fixes it.
**Trac**:
**Username**: larslteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/20607Revise chutney download schedules for exponential backoff2020-06-27T13:18:54ZteorRevise chutney download schedules for exponential backoffCurrently, chutney hard-codes most download schedules to "0, 5".
This should be revised, probably after legacy/trac#20606 is implemented.
Split off legacy/trac#20597.Currently, chutney hard-codes most download schedules to "0, 5".
This should be revised, probably after legacy/trac#20606 is implemented.
Split off legacy/trac#20597.https://gitlab.torproject.org/tpo/core/chutney/-/issues/21000Make chutney select different options depending on the tor version2020-07-23T20:28:17ZteorMake chutney select different options depending on the tor versionSome versions of tor have bugs that chutney needs to work around, for example, legacy/trac#20996 requires tor versions <= 0.3.0.1-alpha to set `UseMicrodescriptors 0` when setting `ClientUseIPv4 0`.
But we want to test newer versions wi...Some versions of tor have bugs that chutney needs to work around, for example, legacy/trac#20996 requires tor versions <= 0.3.0.1-alpha to set `UseMicrodescriptors 0` when setting `ClientUseIPv4 0`.
But we want to test newer versions without this workaround, so we need some kind of versioning feature. (Or test alternatives, where we report more than just pass or failure - but I think that's a different ticket.)https://gitlab.torproject.org/tpo/core/chutney/-/issues/21001Test that recent tor clients can use microdescriptors when configured to only...2020-06-27T13:18:54ZteorTest that recent tor clients can use microdescriptors when configured to only use IPv6Since legacy/trac#20996 was fixed in 0.3.0.1-alpha, we can remove the `UseMicrodescriptors 0` workaround in the chutney IPv6-only tests: but only for recent tor versions (see legacy/trac#21000).Since legacy/trac#20996 was fixed in 0.3.0.1-alpha, we can remove the `UseMicrodescriptors 0` workaround in the chutney IPv6-only tests: but only for recent tor versions (see legacy/trac#21000).teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/21371chutney should display bug warnings logged by tor2020-06-27T13:18:53Zteorchutney should display bug warnings logged by torWhile chutney runs, it should echo any bug messages that any of its running tor instances logs.
We could write a script in tools/ that logs all bug messages until tor exits. Or we could just run a script when chutney finishes.
Perhaps ...While chutney runs, it should echo any bug messages that any of its running tor instances logs.
We could write a script in tools/ that logs all bug messages until tor exits. Or we could just run a script when chutney finishes.
Perhaps we should log unexpected warnings as well.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/21461Require authentication on all tor control ports2020-06-27T13:18:53ZteorRequire authentication on all tor control portsIn legacy/trac#19765, I accidentally made control port authentication only on for clients.In legacy/trac#19765, I accidentally made control port authentication only on for clients.teorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/21462Add a control socket for every tor instance2020-06-27T13:18:53ZteorAdd a control socket for every tor instanceMaybe we'll need to disable this on WindowsMaybe we'll need to disable this on Windowsteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/21463Chutney creates keys directory for clients2020-07-23T20:28:31ZteorChutney creates keys directory for clientsBut clients don't ever use it.But clients don't ever use it.https://gitlab.torproject.org/tpo/core/chutney/-/issues/21464Make relay and client directory permissions consistent2020-06-27T13:18:53ZteorMake relay and client directory permissions consistentChutney creates the data directory with mode 0777, and when relays add keys to it, they modify the permissions to 0700.
But clients never add keys, so they remain at 777.
We can make this consistent by creating the directory with mode ...Chutney creates the data directory with mode 0777, and when relays add keys to it, they modify the permissions to 0700.
But clients never add keys, so they remain at 777.
We can make this consistent by creating the directory with mode 0700 to begin with.teorteor