The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2020-06-27T13:50:00Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30864Move variable manipulation code out of confparse.c2020-06-27T13:50:00ZNick MathewsonMove variable manipulation code out of confparse.cThere is some code in confparse.c for messing with arbitrary variables that ought to become general-purpose and go into lib/. This is a first step towards splitting config.c into reasonable pieces.There is some code in confparse.c for messing with arbitrary variables that ought to become general-purpose and go into lib/. This is a first step towards splitting config.c into reasonable pieces.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30860Add a chutney job that runs on macOS, so that IPv6 chutney tests work2020-06-27T13:50:00ZteorAdd a chutney job that runs on macOS, so that IPv6 chutney tests workTravis Linux doesn't support IPv6, so we should add a macOS chutney job to Tor's CI.
If that's too slow, we can:
* add the chutney tests to the end of an existing macOS job
* change the chutney job to macOS
Remember: each build has a l...Travis Linux doesn't support IPv6, so we should add a macOS chutney job to Tor's CI.
If that's too slow, we can:
* add the chutney tests to the end of an existing macOS job
* change the chutney job to macOS
Remember: each build has a limit of 2 concurrent macOS jobs.
We can do this task after legacy/trac#29280 merges.Tor: 0.2.9.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30859Skip "make test" in Travis stem builds2020-06-27T13:50:00ZteorSkip "make test" in Travis stem buildsOnce legacy/trac#29280 merges, we want to write a separate branch that skips "make test" in stem builds.Once legacy/trac#29280 merges, we want to write a separate branch that skips "make test" in stem builds.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30858Load geoip and geoip6 files during the unit tests2022-06-17T18:57:55ZteorLoad geoip and geoip6 files during the unit testsWe don't run any tests on the contents of the geoip and geoip6 files.
We should write a test that loads the files, and fails if they fail to parse, are empty, or have an unexpected number of entries. (+-25% of the current number of entr...We don't run any tests on the contents of the geoip and geoip6 files.
We should write a test that loads the files, and fails if they fail to parse, are empty, or have an unexpected number of entries. (+-25% of the current number of entries?)
We might want a #define that skips the test, so that people can build and test without geoip. (We might eventually want geoip to be an optional module, but that's out of scope.)
Maybe karsten can tell us how much the number of entries in the geoip files varies?https://gitlab.torproject.org/tpo/core/tor/-/issues/30852Update to June GeoIP2 database2020-06-27T13:50:00ZKarsten LoesingUpdate to June GeoIP2 database[My geoip-2019-06-10 branch](https://gitweb.torproject.org/user/karsten/tor.git/log/?h=geoip-2019-06-10) contains the updated `geoip` and `geoip6` files with IPv4 and IPv6 ranges and is supposed to be merged into maint-0.2.9 and other b...[My geoip-2019-06-10 branch](https://gitweb.torproject.org/user/karsten/tor.git/log/?h=geoip-2019-06-10) contains the updated `geoip` and `geoip6` files with IPv4 and IPv6 ranges and is supposed to be merged into maint-0.2.9 and other branches that are still maintained.Tor: 0.2.9.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30841Allow TOR_MASTER_NAME and TOR_WKT_NAME to be overridden in the git scripts2020-06-27T13:50:00ZteorAllow TOR_MASTER_NAME and TOR_WKT_NAME to be overridden in the git scriptsLet's make all the things configurable.Let's make all the things configurable.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30840Stop hard-coding the bash path in the git scripts2020-06-27T13:50:00ZteorStop hard-coding the bash path in the git scriptsSome OSes (BSD) don't install bash by default. Others (macOS) have an ancient version of bash, which is too old for our git scripts.Some OSes (BSD) don't install bash by default. Others (macOS) have an ancient version of bash, which is too old for our git scripts.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30839Update EndOfLifeTor.md with our latest end of life process2020-06-27T13:50:01ZteorUpdate EndOfLifeTor.md with our latest end of life processThe draft is currently in:
https://github.com/torproject/tor/pull/1062/files
Once legacy/trac#28453 merges, I will update the document with:
* Update git scripts
* Update Tor Travis Cron Jobs instructions
* Jenkins test builds
* Jenkins...The draft is currently in:
https://github.com/torproject/tor/pull/1062/files
Once legacy/trac#28453 merges, I will update the document with:
* Update git scripts
* Update Tor Travis Cron Jobs instructions
* Jenkins test builds
* Jenkins deb builds, including nightlies and experimental-(latest release), which needs to have a successful build before:
* Debian website instructions, including JavaScript and no script lists
* Chutney Travis CI
* Stem Travis CI
* sbws Travis CI (the sbws CI does not use multiple tor versions yet, so add a ticket number)Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30835Tor's maint-0.3.4 branch is deprecated, and maint-0.4.1 has been created2020-06-27T13:50:01ZteorTor's maint-0.3.4 branch is deprecated, and maint-0.4.1 has been createdParent task for 0.3.4 deprecation and 0.4.1 addition.Parent task for 0.3.4 deprecation and 0.4.1 addition.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30831Separate front-end and back-end of handle implementation.2020-06-27T13:50:01ZNick MathewsonSeparate front-end and back-end of handle implementation.As part of legacy/trac#29218, we should separate the interface and implementation parts of our handle code, so we can support multiple implementations.As part of legacy/trac#29218, we should separate the interface and implementation parts of our handle code, so we can support multiple implementations.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30828Design tests for as-yet-untested controller commands in Stem2022-06-17T13:51:30ZNick MathewsonDesign tests for as-yet-untested controller commands in StemIn legacy/trac#30676, we inventoried the controller commands that don't currently get testing by stem, even with the ONLINE tests. They are:
```
dropguards
dropownership
hspost
mapaddress
postdescriptor
redirectstream
resolve
```
Let'...In legacy/trac#30676, we inventoried the controller commands that don't currently get testing by stem, even with the ONLINE tests. They are:
```
dropguards
dropownership
hspost
mapaddress
postdescriptor
redirectstream
resolve
```
Let's describe a set of tests for each, and open tickets for adding them to stem.https://gitlab.torproject.org/tpo/core/tor/-/issues/30825Travis: remove cron jobs for 0.3.4; add for 0.4.12020-06-27T13:50:01ZNick MathewsonTravis: remove cron jobs for 0.3.4; add for 0.4.10.3.4 has reached EOL. 0.4.1 now exists as a separate branch. We should update our cron jobs accordingly.
(I don't actually know how our cron jobs are set up; it is possible this is a no-op.)0.3.4 has reached EOL. 0.4.1 now exists as a separate branch. We should update our cron jobs accordingly.
(I don't actually know how our cron jobs are set up; it is possible this is a no-op.)Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30822Update git scripts to remove 0.3.4, include 0.4.12020-06-27T13:50:01ZNick MathewsonUpdate git scripts to remove 0.3.4, include 0.4.10.3.4 is now EOL; it should no longer be covered by our push-all/pull-all/merge-forward scripts.
0.4.1 is now a separate branch; it should new be covered by those scripts.0.3.4 is now EOL; it should no longer be covered by our push-all/pull-all/merge-forward scripts.
0.4.1 is now a separate branch; it should new be covered by those scripts.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30821Fix a typo in test_rebind.sh2020-06-27T13:50:01ZteorFix a typo in test_rebind.shTor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30817Write a proposal for tor bootstrapping that works on slow links, but avoids s...2022-06-17T14:13:14ZteorWrite a proposal for tor bootstrapping that works on slow links, but avoids slow relaysIn legacy/trac#16844, clients on slow links time out before they can download a full consensus
In legacy/trac#21969 and children, relays fail to bootstrap because a directory authority limits DirPort download speeds (a similar bug exist...In legacy/trac#16844, clients on slow links time out before they can download a full consensus
In legacy/trac#21969 and children, relays fail to bootstrap because a directory authority limits DirPort download speeds (a similar bug exists for clients which try slow relays)
We need to redesign tor's bootstrap so that tor works when the link is slow, but tries another relay when the relay is slow.
We should implement multiple concurrent downloads for all directory documents, not just consensuses. Once we have multiple concurrent downloads, we can increase the timeouts substantially.
We should limit the number of concurrent downloads to 3, because if 3 fast relays are all slow, it's probably the link that it slow. And a 3x download size is an acceptable cost. (It probably won't be that bad, because we delay starting the 2nd and 3rd fetches, and terminate them when the first one completes.)
We could also be a bit more clever, and terminate the download that would take the longest time to finish, after a soft timeout. And then reduce the number of concurrent downloads by one.
We could also simplify the relay/authority selection logic:
* relays try authorities first, then after a delay, they try other relays (most relays are directory mirrors, so they do this anyway)
* clients try relays first, then after a delay, they try authorities
And simplify the ORPort/DirPort selection logic for directory downloads:
* clients always download via ORPorts on relays and authorities, for security
* relays always download via DirPorts on authorities, to avoid SSL CPU load on a small number of machines
* relays always download via ORPorts on other relays, for security (CPU load doesn't matter that much)
We could design a new directory download module to implement this logic, using pieces of the existing modules, but with a cleaner, high-level interface:
* request a download of a particular directory document, or set of directory documents
* pass a download configuration with:
* an optional directory cache,
* ORPort / DirPort preference,
* number of permitted concurrent connections,
* relay and authority initial delays,
* status and completion handlers.https://gitlab.torproject.org/tpo/core/tor/-/issues/30816Remove ping ::1 from tor's test-network-all and simplify the logic2021-09-16T14:23:38ZteorRemove ping ::1 from tor's test-network-all and simplify the logicAfter legacy/trac#30459 is merged into chutney, and everyone updates, we can remove the complicated checks for IPv6 from tor's test-network-all.After legacy/trac#30459 is merged into chutney, and everyone updates, we can remove the complicated checks for IPv6 from tor's test-network-all.https://gitlab.torproject.org/tpo/core/tor/-/issues/30813Fails to build with the latest libevent-2.1.10: Undefined symbol "evutil_secu...2020-06-27T13:50:02Zyurivict271Fails to build with the latest libevent-2.1.10: Undefined symbol "evutil_secure_rng_add_bytes"Undefined symbol "evutil_secure_rng_add_bytes"
See the FreeBSD bug report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238433Undefined symbol "evutil_secure_rng_add_bytes"
See the FreeBSD bug report: https://bugs.freebsd.org/bugzilla/show_bug.cgi?id=238433https://gitlab.torproject.org/tpo/core/tor/-/issues/30806Make a subsystem for evloop2020-06-27T13:50:02ZNick MathewsonMake a subsystem for evloopThe evloop module should use the subsystem mechanism.The evloop module should use the subsystem mechanism.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30804util/socketpair_ersatz test requires configured network2020-06-27T13:50:02ZNick Mathewsonutil/socketpair_ersatz test requires configured networkWhen I use `unshare -r -n ./src/test/test` to try to reproduce legacy/trac#30409, I get this error:
```
util/socketpair_ersatz: [forking]
FAIL src/test/test_util.c:5402: assert(0 OP_EQ socketpair_result): 0 vs -101
[socketpair_ersa...When I use `unshare -r -n ./src/test/test` to try to reproduce legacy/trac#30409, I get this error:
```
util/socketpair_ersatz: [forking]
FAIL src/test/test_util.c:5402: assert(0 OP_EQ socketpair_result): 0 vs -101
[socketpair_ersatz FAILED]
```Tor: 0.4.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30801Investigate running CI with hardened dependencies vs running CI with valgrind2022-06-17T18:06:43ZNick MathewsonInvestigate running CI with hardened dependencies vs running CI with valgrindIn legacy/trac#30674, we investigated why running with --enable-fragile-hardening had missed a memory leak that valgrind could successfully catch. The answer turned out to be that we had not compiled our dependencies with sanitizers ena...In legacy/trac#30674, we investigated why running with --enable-fragile-hardening had missed a memory leak that valgrind could successfully catch. The answer turned out to be that we had not compiled our dependencies with sanitizers enabled -- so they didn't catch memory leaks that happened inside our dependencies.
Assuming we want CI to catch this kind of bug (and we do!) the alternatives seem to be: build our dependencies with sanitizers, or run with valgrind.
Teor made the following notes about deployment and evaluations:
> Hardened dependencies:
> 1. We know we can harden dependencies
> 2. Hardened dependencies may cause CI failures due to bugs in dependencies
> 3. Hardened dependencies may be slower
> 4. We probably won't rebuild libc and other large libraries in hardened mode
> 5. We don't know if valgrind or hardened builds provide better coverage of the kinds of coding errors we typically make
> 6. It might be complicated to configure builds for all our dependencies
> 7. We can't harden our chutney, stem, and sbws CIs, because they use pre-built binaries
>
> Valgrind:
> 1. We don't know if valgrind runs well in Travis CI
> 2. Valgrind may cause CI failures due to bugs in dependencies
> 3. Valgrind may be slower
> 4. Valgrind instruments all the code, no matter which library it's in
> 5. We don't know if valgrind or hardened builds provide better coverage of the kinds of coding errors we typically make
> 6. Valgrind is simple to configure
> 7. We can run valgrind on the pre-built binaries in our chutney, stem, and sbws CIs
We should come to a decision here and take action.