Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T13:29:30Zhttps://gitlab.torproject.org/legacy/trac/-/issues/17002chutney start could tell users how to kill old chutney tors2020-06-13T13:29:30Zteorchutney start could tell users how to kill old chutney torsThe most common reason chutney start fails to start all tor processes is port conflicts caused by old chutney tor processes still running.
We could tell users how to kill processes (should we?) using a summary of the following informati...The most common reason chutney start fails to start all tor processes is port conflicts caused by old chutney tor processes still running.
We could tell users how to kill processes (should we?) using a summary of the following information from https://trac.torproject.org/projects/tor/wiki/doc/TorChutneyGuide
"However, sometimes chutney fails to launch due to a persistent error. The most common issue is having a previous chutney network active. Old chutney networks are typically killed by chutney/tools/bootstrap-network.sh using chutney stop, but sometimes this doesn't happen. The tor processes launched by chutney can be found using a command like `ps auxwww | grep chutney | grep tor`. Ensure each pid doesn't belong to a process you want, like a system tor or an unrelated process, then kill each chutney tor pid using `kill`. Alternately, each tor instance's pid files can be found in the chutney/net/nodes directory. (Old nodes directories are renamed using an increasing numeric timestamp as a suffix.)"
Use:
* chutney stop
* `ps auxwww | grep chutney | grep tor` and carefull check each pid before using `kill` on it
* the tor pid files in the chutney/net/nodes* directories (maybe, these can be very old)https://gitlab.torproject.org/legacy/trac/-/issues/16950Add a chutney command that checks if network bootstrap has finished2020-06-13T14:48:41ZteorAdd a chutney command that checks if network bootstrap has finishedThis command ideally needs to check (or wait before returning until) all clients are ready to send to all endpoints:
* Authorities: consensus is working
* Other Nodes have a copy of the consensus
* Hidden Services have uploaded descripto...This command ideally needs to check (or wait before returning until) all clients are ready to send to all endpoints:
* Authorities: consensus is working
* Other Nodes have a copy of the consensus
* Hidden Services have uploaded descriptors
* Anything else?https://gitlab.torproject.org/legacy/trac/-/issues/16949Make Chutney Easier to Use & More Functional, Write Guide2020-06-13T13:29:27ZteorMake Chutney Easier to Use & More Functional, Write GuideI (teor) am writing a guide to using chuntney.
As part of that, I'm making chuntey easier to use, mainly through a series of tweaks that simplify usage and improve test coverage.
See https://trac.torproject.org/projects/tor/wiki/doc/Tor...I (teor) am writing a guide to using chuntney.
As part of that, I'm making chuntey easier to use, mainly through a series of tweaks that simplify usage and improve test coverage.
See https://trac.torproject.org/projects/tor/wiki/doc/TorChutneyGuide and the child tickets for more details.https://gitlab.torproject.org/legacy/trac/-/issues/16811Mark TestingTorNetwork PREDICT_UNLIKELY2020-06-13T14:48:11ZteorMark TestingTorNetwork PREDICT_UNLIKELYI'd like to see every instance of:
`if (options->TestingTorNetwork) {` (or similarly, `get_options()->`)
transformed to:
`if (PREDICT_UNLIKELY(options->TestingTorNetwork)) {`
This should give us a slight performance boost in relays and ...I'd like to see every instance of:
`if (options->TestingTorNetwork) {` (or similarly, `get_options()->`)
transformed to:
`if (PREDICT_UNLIKELY(options->TestingTorNetwork)) {`
This should give us a slight performance boost in relays and authorities, at the cost of a slight slowdown due to missed branch predictions in test networks.
This would be best done using a coccinelle, perl, or sed script.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16809High coverage on node/path selection functions2020-06-13T14:48:10ZNick MathewsonHigh coverage on node/path selection functionsThese functions are stupidly fragile, since bad behavior tends to happen only under rare circumstances. We should make sure that, through integration and unit tests, we hit all the cases. Note that coverage isn't enough if we don't chec...These functions are stupidly fragile, since bad behavior tends to happen only under rare circumstances. We should make sure that, through integration and unit tests, we hit all the cases. Note that coverage isn't enough if we don't check to make sure that the statistical distribution on paths is good, and the constraints are actually obeyed.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16808High coverage on connection_edge, addressmap2020-06-13T14:48:09ZNick MathewsonHigh coverage on connection_edge, addressmapThese functions get more than their share of bugs, and aren't very well reached by our current tests. We can use integration and unit tests for these.These functions get more than their share of bugs, and aren't very well reached by our current tests. We can use integration and unit tests for these.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16806Chutney/integration tests for DNS server/client functionality2020-06-13T13:29:24ZNick MathewsonChutney/integration tests for DNS server/client functionalityhttps://gitlab.torproject.org/legacy/trac/-/issues/16805Improve unit-test coverage on old and/or pure-ish functions/modules in src/or2020-06-13T14:48:08ZNick MathewsonImprove unit-test coverage on old and/or pure-ish functions/modules in src/orThese files have the most old code in src/or right now, and they have a lot of functions that are mostly pure-ish functions. Let's call these low-hanging fruit.
```
routerparse.c.gcov 717 1620 69.32
routerlist.c.gcov 1496 588 28.21
node...These files have the most old code in src/or right now, and they have a lot of functions that are mostly pure-ish functions. Let's call these low-hanging fruit.
```
routerparse.c.gcov 717 1620 69.32
routerlist.c.gcov 1496 588 28.21
nodelist.c.gcov 503 165 24.70
rendcache.c.gcov 332 1 0.30
networkstatus.c.gcov 681 144 17.45
policies.c.gcov 244 506 67.47
rephist.c.gcov 931 278 22.99
buffers.c.gcov 361 506 58.36
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16801Increase torgzip coverage as high as possible2020-06-13T14:48:07ZNick MathewsonIncrease torgzip coverage as high as possibleWe've actually had bugs in this one that caused needless bandwidth wastage. (eg #11824 and #11787.) Let's see if there are any more!
```
x/torgzip.c.gcov 88 136 60.71
```We've actually had bugs in this one that caused needless bandwidth wastage. (eg #11824 and #11787.) Let's see if there are any more!
```
x/torgzip.c.gcov 88 136 60.71
```Tor: 0.2.9.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/16799Raise utility testing over 95%2020-06-13T14:48:06ZNick MathewsonRaise utility testing over 95%These modules perform utility functionality that we should be able to get to a very high testing coverage rate.
```
x/memarea.c.gcov 20 85 80.95
x/util.c.gcov 484 1263 72.30
x/util_format.c.gcov 11 183 94.33
x/workqueue.c.gcov 34 117 77...These modules perform utility functionality that we should be able to get to a very high testing coverage rate.
```
x/memarea.c.gcov 20 85 80.95
x/util.c.gcov 484 1263 72.30
x/util_format.c.gcov 11 183 94.33
x/workqueue.c.gcov 34 117 77.48
x/address.c.gcov 77 595 88.54
x/log.c.gcov 274 274 50.00
```
(In the case of one or two of these, they might be actually compatibility stuff. In which case, compatibility stuff should move into new compat_* modules to be considered under #16798)Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16798Raise compat_* testing to over 80%2020-06-13T14:58:35ZNick MathewsonRaise compat_* testing to over 80%```
compat.c.gcov 360 426 54.20
compat_libevent.c.gcov 92 32 25.81
compat_pthreads.c.gcov 22 72 76.60
compat_threads.c.gcov 48 51 51.52
```
Getting 95% coverage for stuff that relies on OS functionality might not be feasible, but it wou...```
compat.c.gcov 360 426 54.20
compat_libevent.c.gcov 92 32 25.81
compat_pthreads.c.gcov 22 72 76.60
compat_threads.c.gcov 48 51 51.52
```
Getting 95% coverage for stuff that relies on OS functionality might not be feasible, but it would be great to get this as high as possible.
(Stubbing OS calls out is probably *not* the best answer here, even though you might think it would be. We're not only interested in whether our compatibility wrappers do the right thing when the OS behaves the way we expect: we're also interested in whether they do the right thing on the actual OS. )Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16795Process-related utility code should have unit test coverage over 95%2020-06-13T14:48:04ZNick MathewsonProcess-related utility code should have unit test coverage over 95%```
x/procmon.c.gcov 45 0 0.00
x/util_process.c.gcov 5 33 86.84
``````
x/procmon.c.gcov 45 0 0.00
x/util_process.c.gcov 5 33 86.84
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16794All cryptography unit test coverage should be over 95%; all should have test ...2020-06-13T14:57:00ZNick MathewsonAll cryptography unit test coverage should be over 95%; all should have test vectorsIt's high, but it's not there yet:
```
x/crypto.c.gcov 204 738 78.34
x/crypto_curve25519.c.gcov 8 74 90.24
x/crypto_ed25519.c.gcov 17 95 84.82
x/crypto_format.c.gcov 5 95 95.00
x/crypto_pwbox.c.gcov 2 59 96.72
x/crypto_s2k.c.gcov 14 152...It's high, but it's not there yet:
```
x/crypto.c.gcov 204 738 78.34
x/crypto_curve25519.c.gcov 8 74 90.24
x/crypto_ed25519.c.gcov 17 95 84.82
x/crypto_format.c.gcov 5 95 95.00
x/crypto_pwbox.c.gcov 2 59 96.72
x/crypto_s2k.c.gcov 14 152 91.57
x/aes.c.gcov 0 17 100.00
TOTAL 250 1230 83.11
```Tor: 0.2.9.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/16792Have a way to mark lines as "unreachable by unit tests"2020-06-13T15:15:02ZNick MathewsonHave a way to mark lines as "unreachable by unit tests"It would be great to have a way to tell the test coverage code "this line won't be reached by tests, so don't worry about it." This would let us have a prayer of reaching 100%.It would be great to have a way to tell the test coverage code "this line won't be reached by tests, so don't worry about it." This would let us have a prayer of reaching 100%.Tor: 0.2.9.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/16791All modules in src/common should have 90-95%+ unit test coverage2020-06-13T14:48:01ZNick MathewsonAll modules in src/common should have 90-95%+ unit test coverageIt's a dirty shame to have anything in src/common that isn't really deeply covered by our unit tests, since src/common is supposed to be full of stuff that's _easy_ to test.
As of 2015-08-13, here's what's problematic (running "make che...It's a dirty shame to have anything in src/common that isn't really deeply covered by our unit tests, since src/common is supposed to be full of stuff that's _easy_ to test.
As of 2015-08-13, here's what's problematic (running "make check") on OSX, sorted by percentage of lines covered.
```
x/procmon.c.gcov 45 0 0.00
x/sandbox.c.gcov 9 0 0.00
x/compat_libevent.c.gcov 92 32 25.81
x/tortls.c.gcov 593 355 37.45
x/log.c.gcov 274 274 50.00
x/compat_threads.c.gcov 48 51 51.52
x/compat.c.gcov 360 426 54.20
x/torgzip.c.gcov 88 136 60.71
x/util.c.gcov 484 1263 72.30
x/compat_pthreads.c.gcov 22 72 76.60
x/workqueue.c.gcov 34 117 77.48
x/crypto.c.gcov 204 738 78.34
x/memarea.c.gcov 20 85 80.95
x/backtrace.c.gcov 9 42 82.35
x/crypto_ed25519.c.gcov 17 95 84.82
x/util_process.c.gcov 5 33 86.84
x/address.c.gcov 77 595 88.54
x/crypto_curve25519.c.gcov 8 74 90.24
x/crypto_s2k.c.gcov 14 152 91.57
x/util_format.c.gcov 11 183 94.33
x/crypto_format.c.gcov 5 95 95.00
x/container.c.gcov 15 390 96.30
x/crypto_pwbox.c.gcov 2 59 96.72
x/di_ops.c.gcov 1 37 97.37
x/aes.c.gcov 0 17 100.00
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16764Simplify Tor's control flow graph to the extent we can.2020-06-13T15:32:14ZNick MathewsonSimplify Tor's control flow graph to the extent we can.For background, see http://archives.seul.org/tor/dev/Mar-2015/msg00197.html .
As of this writing, the largest strongly-connected component in Tor's callgraph has 407 functions in it. Nobody can actually understand a program that's so c...For background, see http://archives.seul.org/tor/dev/Mar-2015/msg00197.html .
As of this writing, the largest strongly-connected component in Tor's callgraph has 407 functions in it. Nobody can actually understand a program that's so complex. Let's simplify it!
(This is a parent ticket.)Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16698Split directory_handle_command_get into per-command functions2020-06-13T14:47:45ZNick MathewsonSplit directory_handle_command_get into per-command functionsTor: 0.2.9.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/16454chutney's "tor does not support this option" is unclear2020-06-13T13:29:23Zteorchutney's "tor does not support this option" is unclearA user emailed me concerned that chutney was giving a lot of "tor does not support this option" errors.
These messages could be much clearer:
* The message is a warning, not an error
* The message doesn't describe the likely consequence...A user emailed me concerned that chutney was giving a lot of "tor does not support this option" errors.
These messages could be much clearer:
* The message is a warning, not an error
* The message doesn't describe the likely consequences. We could document these, at least for the 3 `TestingDirAuthVote*` options
I also wrote back to them:
```
This is a warning that happens when you have an older version of tor. It is telling you that your version of tor doesn't support one of the options used in the standard chutney torrc templates. Chutney warns you, but disables the option so that tor will still run.
This particular option is used to speed up the authorities' votes for hidden service directories. If you are running a chutney network with a hidden service, you can speed up bootstrap by using tor 0.2.6.9 or tor 0.2.7-alpha.
If you're not using hidden services, you can disregard this warning.
Thank you for letting me know, I will log a task to update the documentation to make it clearer that this is a warning, and explain the likely consequence.
```https://gitlab.torproject.org/legacy/trac/-/issues/15784Clarify Chutney exit policies for IPv4 and IPv6, including private addresses2020-06-13T13:29:19ZteorClarify Chutney exit policies for IPv4 and IPv6, including private addressesAllow chutney exits to exit to all private addresses.
Tidy up exit policies, so each part of the policy is clearly identified.
Document alternatives to work around #11264, the microdescriptor 2x /8 requirement for exits.
Prepare for r...Allow chutney exits to exit to all private addresses.
Tidy up exit policies, so each part of the policy is clearly identified.
Document alternatives to work around #11264, the microdescriptor 2x /8 requirement for exits.
Prepare for resolving #15353 by making sure the exit policies *should* work when localhost is the only available IP. This still requires further investigation.
Patch:
**Branch:** clarify-exit-policies
**Repository:** ​https://github.com/teor2345/chutney.gitteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/15783On chutney verify failure, inform user about debug mode2020-06-13T13:29:18ZteorOn chutney verify failure, inform user about debug modeWhen chutney verify fails, tell the user how to activate debug mode by editing the source code.
Eventually, we'll make debug mode an argument, see #15418.When chutney verify fails, tell the user how to activate debug mode by editing the source code.
Eventually, we'll make debug mode an argument, see #15418.teorteor