The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2021-09-16T14:23:38Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/31263More tests for practracker2021-09-16T14:23:38ZNick MathewsonMore tests for practrackerI think some integration tests here would make things easier to hack on.I think some integration tests here would make things easier to hack on.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/31202Refactor practracker's issue-listing code to return a generator2021-09-16T14:23:38ZNick MathewsonRefactor practracker's issue-listing code to return a generatorRight now, this issue-listing code exposes the problems with "print()". Instead, it should yield them as a generator, so that the invoking code can decide what to do with them.Right now, this issue-listing code exposes the problems with "print()". Instead, it should yield them as a generator, so that the invoking code can decide what to do with them.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/31176Teach practracker about .may_include files2021-09-16T14:23:38ZNick MathewsonTeach practracker about .may_include filesWe would like to introduce a second category of .may_include rules: those that should only apply on an advisory basis. We would treat violations of these rules as a best practices violation rather than an error. It would allow us to st...We would like to introduce a second category of .may_include rules: those that should only apply on an advisory basis. We would treat violations of these rules as a best practices violation rather than an error. It would allow us to start ratcheting down the number of layering violations.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/31156Add support of TBytes keyword to torrc for AccountingMax setting (and maybe o...2020-07-30T20:37:37ZTracAdd support of TBytes keyword to torrc for AccountingMax setting (and maybe others)Happening on Debian 9 with Tor 0.2.9.16 from the standard apt repo.
(Sorry, if this is already resolved in newer versions. In that case just close and ignore.)
Syslog output:
```
Jul 14 10:50:38 systemd[1]: Starting Anonymizing overla...Happening on Debian 9 with Tor 0.2.9.16 from the standard apt repo.
(Sorry, if this is already resolved in newer versions. In that case just close and ignore.)
Syslog output:
```
Jul 14 10:50:38 systemd[1]: Starting Anonymizing overlay network for TCP...
Jul 14 10:50:39 tor[530]: Jul 14 10:50:39.082 [notice] Tor 0.2.9.16 (git-9ef571339967c1e5) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.1.0k and Zlib 1.2.8.
Jul 14 10:50:39 tor[530]: Jul 14 10:50:39.082 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jul 14 10:50:39 tor[530]: Jul 14 10:50:39.082 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jul 14 10:50:39 tor[530]: Jul 14 10:50:39.082 [notice] Read configuration file "/etc/tor/torrc".
Jul 14 10:50:39 tor[530]: Jul 14 10:50:39.085 [warn] Unknown unit 'TBytes'.
Jul 14 10:50:39 tor[530]: Jul 14 10:50:39.085 [warn] Failed to parse/validate config: Value 'AccountingMax 5 TBytes' is malformed or out of bounds.
Jul 14 10:50:39 tor[530]: Jul 14 10:50:39.085 [err] Reading config failed--see warnings above.
Jul 14 10:50:39 systemd[1]: tor@default.service: Control process exited, code=exited status=1
Jul 14 10:50:39 systemd[1]: Failed to start Anonymizing overlay network for TCP.
```
Additionally, please note that /var/tor/log didn't give any hint of this, instead it went to syslog. Suboptimal.
**Trac**:
**Username**: tlahttps://gitlab.torproject.org/tpo/core/tor/-/issues/30920Detect uint64 overflow in config_parse_units()2020-06-27T13:49:58ZNick MathewsonDetect uint64 overflow in config_parse_units()The config_parse_units function uses 64-bit arithmetic, but does not detect 64-bit overflow. This means that values like "20000000 TB", which should be rejected, are instead mis-parsed.
Since this function is only used for configuratio...The config_parse_units function uses 64-bit arithmetic, but does not detect 64-bit overflow. This means that values like "20000000 TB", which should be rejected, are instead mis-parsed.
Since this function is only used for configuration parsing, it's not a huge issue, but it should be simple enough to resolve this.
Found while working on 30893.Tor: 0.4.3.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30782Cache outdated fetched descriptors on directory authorities2020-06-27T13:50:03ZteorCache outdated fetched descriptors on directory authoritiesWhen tor starts fetching a descriptor due to a consensus, then replaces its consensus before the descriptor arrives, the descriptor is unrecognised.
Directory caches keep the descriptor in the old descriptor cache, but clients and autho...When tor starts fetching a descriptor due to a consensus, then replaces its consensus before the descriptor arrives, the descriptor is unrecognised.
Directory caches keep the descriptor in the old descriptor cache, but clients and authorities drop it.
But I think authorities should keep the descriptor, in case it is used in a future vote or consensus.
This is a bug on tor 0.1.1.13-alpha.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30780Return a distinct was_router_added_t when formatting annotations fails2021-09-16T14:23:38ZteorReturn a distinct was_router_added_t when formatting annotations failsThere's a note in dirserv_add_multiple_descriptors() that this is bad.
But no-one ever fixed it.There's a note in dirserv_add_multiple_descriptors() that this is bad.
But no-one ever fixed it.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30642Add an ed25519-identity file to the data directory2022-03-31T17:09:18ZteorAdd an ed25519-identity file to the data directoryRelays print their RSA fingerprint to a "fingerprint" file in their data directory.
We need an equivalent file for base-64 encoded ed25519 public keys.Relays print their RSA fingerprint to a "fingerprint" file in their data directory.
We need an equivalent file for base-64 encoded ed25519 public keys.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/30630Put CI URLs in ReleasingTor.md2021-07-22T16:19:44ZteorPut CI URLs in ReleasingTor.mdarma asked us where the CI builders are.
The list is in ReleasingTor.md and on the wiki, but only the wiki has the URLs:
https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/CIFailures#CIBuildersarma asked us where the CI builders are.
The list is in ReleasingTor.md and on the wiki, but only the wiki has the URLs:
https://trac.torproject.org/projects/tor/wiki/org/teams/NetworkTeam/CIFailures#CIBuildersTor: 0.4.1.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30455Improve documentation for chutney warnings in "make test-network-all"2021-07-22T16:19:43ZNick MathewsonImprove documentation for chutney warnings in "make test-network-all"It appears that in fb32c522320430f, we added a second call to test-network.sh inside our test-network-all loop. Now the code looks like this:
```
for f in $$flavors; do \
$(SHELL) $(top_srcdir)/test-driver --test-name $$f --log-file...It appears that in fb32c522320430f, we added a second call to test-network.sh inside our test-network-all loop. Now the code looks like this:
```
for f in $$flavors; do \
$(SHELL) $(top_srcdir)/test-driver --test-name $$f --log-file $(TEST_NETWORK_ALL_LOG_DIR)/$$f.log --trs-file $(TEST_NETWORK_ALL_LOG_DIR)/$$f.trs $(TEST_NETWORK_ALL_DRIVER_FLAGS) $(top_srcdir)/src/test/test-network.sh --flavor $$f $(TEST_NETWORK_FLAGS); \
$(top_srcdir)/src/test/test-network.sh $(TEST_NETWORK_WARNING_FLAGS); \
done; \
```
I might be wrong, but it looks to me like we're calling test-network.sh twice in each loop: once through `test-driver`, and once directly.
I'm not going to work on this till teor is back, though, since there are dragons here that I do not understand.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30309Rename tor_mem_is_zero to fast_mem_is_zero; use it consistently2021-09-16T14:19:58ZNick MathewsonRename tor_mem_is_zero to fast_mem_is_zero; use it consistentlyBy convention, tor_memeq() means "constant time" and fast_memeq() means "variable-time." But tor_mem_is_zero is variable-time, whereas its constant-time variant is safe_mem_is_zero.
I'm fine with the "safe_" name, but we should rename ...By convention, tor_memeq() means "constant time" and fast_memeq() means "variable-time." But tor_mem_is_zero is variable-time, whereas its constant-time variant is safe_mem_is_zero.
I'm fine with the "safe_" name, but we should rename tor_mem_is_zero() to indicate that it is the fast one, not the safe one. We should also audit its users, particularly tor_digest{256,}_is_zero()Tor: 0.4.1.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30267Simplify the code logic of launching HS circuits2021-09-16T14:19:58ZGeorge KadianakisSimplify the code logic of launching HS circuitsThe intro/rendezvous parts of `circuit_get_open_circ_or_launch()` are very complicated, and then they call `circuit_get_open_circ_or_launch()` which is also extremely complicated.
A minimal action item for improving the situation here w...The intro/rendezvous parts of `circuit_get_open_circ_or_launch()` are very complicated, and then they call `circuit_get_open_circ_or_launch()` which is also extremely complicated.
A minimal action item for improving the situation here would be to split the HS-parts of `connection_ap_handshake_attach_circuit()` which are already pretty disentangled into their own function. That's pretty easy to do.
The harder part of this would be to see if we can also split the HS parts of `circuit_get_open_circ_or_launch()` in some way.https://gitlab.torproject.org/tpo/core/tor/-/issues/30187100% cpu usage in winthreads tor_cond_wait2021-01-19T21:23:07ZTrac100% cpu usage in winthreads tor_cond_waitFor years I run relay using self-compiled win64 version of tor.
Compiler mingw64.
Relay runs well for some time but suddenly starts using 100% cpu all cores.
I traced where it happens. The following loop never ends :
```
do {
DWO...For years I run relay using self-compiled win64 version of tor.
Compiler mingw64.
Relay runs well for some time but suddenly starts using 100% cpu all cores.
I traced where it happens. The following loop never ends :
```
do {
DWORD res;
res = WaitForSingleObject(cond->event, ms);
EnterCriticalSection(&cond->lock);
if (cond->n_to_wake &&
cond->generation != generation_at_start) {
--cond->n_to_wake;
--cond->n_waiting;
result = 0;
waiting = 0;
goto out;
} else if (res != WAIT_OBJECT_0) {
result = (res==WAIT_TIMEOUT) ? 1 : -1;
--cond->n_waiting;
waiting = 0;
goto out;
} else if (ms != INFINITE) {
endTime = GetTickCount();
if (startTime + ms_orig <= endTime) {
result = 1; /* Timeout */
--cond->n_waiting;
waiting = 0;
goto out;
} else {
ms = startTime + ms_orig - endTime;
}
}
/* If we make it here, we are still waiting. */
if (cond->n_to_wake == 0) {
/* There is nobody else who should wake up; reset
* the event. */
ResetEvent(cond->event);
}
out:
LeaveCriticalSection(&cond->lock);
} while (waiting);
```
res = WAIT_OBJECT_0;
ms = INFINITE;
cond->n_to_wake=0x11
cond->generation=0x28
generation_at_start=0x28
it means no path with "goto out" ever execute
more than one thread run this loop and each one eat separate core
Some people I shared binaries with report same problem.
Pls check
**Trac**:
**Username**: bolvanTor: 0.4.4.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30119cert-spec uses binary encodings but does not specify byte order2021-07-22T16:19:43Zirlcert-spec uses binary encodings but does not specify byte orderFrom looking at the Tor implementation (in a black box way, didn't look at the source code) these seem to be big endian byte order. We should have a note in cert-spec for implementors so they don't have to guess.From looking at the Tor implementation (in a black box way, didn't look at the source code) these seem to be big endian byte order. We should have a note in cert-spec for implementors so they don't have to guess.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30109Document that MapAddress is automatically strict, but does not handle redirects2021-08-23T15:16:06ZteorDocument that MapAddress is automatically strict, but does not handle redirectsWe should document that:
1. StrictNodes does not apply to MapAddress
2. MapAddress ~~falls back to a random exit by default~~ fails rather than falling back to a random exit
Edited to add:
3. If the site does a redirect, MapAddress does...We should document that:
1. StrictNodes does not apply to MapAddress
2. MapAddress ~~falls back to a random exit by default~~ fails rather than falling back to a random exit
Edited to add:
3. If the site does a redirect, MapAddress does not apply to the new siteTor: 0.4.0.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30045output of "tor --key-expiration sign" should be a time stamp2020-08-04T17:29:41Ztoralfoutput of "tor --key-expiration sign" should be a time stampIt would be helpful for a cron job having sth like
```
let "diff = $(tor --key-expiration sign --format=timestamp) - $(date +%s)"
```
in it.It would be helpful for a cron job having sth like
```
let "diff = $(tor --key-expiration sign --format=timestamp) - $(date +%s)"
```
in it.https://gitlab.torproject.org/tpo/core/tor/-/issues/29979Don't recommend expect() in CodingStandardsRust, because it panics2021-07-22T16:19:43ZteorDon't recommend expect() in CodingStandardsRust, because it panicsSee the pull request from a GitHub contributor:
https://github.com/torproject/tor/pull/883See the pull request from a GitHub contributor:
https://github.com/torproject/tor/pull/883Tor: 0.4.1.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29789practracker.py codec exception in some locales2020-06-27T13:50:39ZTaylor Yupractracker.py codec exception in some localespractracker.py, implemented in legacy/trac#29221, seems to have a locale dependency when python3 is being used. If the locale isn't a UTF-8 locale, UTF-8 characters in sources can result in an exception:
```
$ LANG=en_US.US-ASCII make ...practracker.py, implemented in legacy/trac#29221, seems to have a locale dependency when python3 is being used. If the locale isn't a UTF-8 locale, UTF-8 characters in sources can result in an exception:
```
$ LANG=en_US.US-ASCII make check-best-practices PYTHON=python
python ../scripts/maint/practracker/practracker.py ..
mirkwood:build-norust tlyu$ LANG=en_US.US-ASCII make check-best-practices
python3 ../scripts/maint/practracker/practracker.py ..
Traceback (most recent call last):
File "../scripts/maint/practracker/practracker.py", line 151, in <module>
main()
File "../scripts/maint/practracker/practracker.py", line 134, in main
found_new_issues = consider_all_metrics(files_list)
File "../scripts/maint/practracker/practracker.py", line 89, in consider_all_metrics
found_new_issues |= consider_metrics_for_file(fname, f)
File "../scripts/maint/practracker/practracker.py", line 104, in consider_metrics_for_file
found_new_issues |= consider_file_size(fname, f)
File "../scripts/maint/practracker/practracker.py", line 51, in consider_file_size
file_size = metrics.get_file_len(f)
File "/Users/tlyu/src/tor/scripts/maint/practracker/metrics.py", line 11, in get_file_len
for i, l in enumerate(f):
File "/Users/tlyu/src/brew/Cellar/python/3.7.2_2/Frameworks/Python.framework/Versions/3.7/lib/python3.7/encodings/ascii.py", line 26, in decode
return codecs.ascii_decode(input, self.errors)[0]
UnicodeDecodeError: 'ascii' codec can't decode byte 0xc3 in position 14: ordinal not in range(128)
make: *** [check-best-practices] Error 1
```
I'm also seeing this on gitlab.com CI, but I don't know offhand what its locale environment variables are.
We might want to use the `encoding=` keyword parameter to `open()`, but I think that would no longer be python2 compatible.Tor: 0.4.1.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29746Improve Tor best practices tracker2021-09-16T14:24:10ZGeorge KadianakisImprove Tor best practices trackerVarious improvements could be done to the best-practices tracker (practracker legacy/trac#29221). Here are a few from Nick's last review:
https://github.com/torproject/tor/pull/742#discussion_r262564420
https://github.com/torproject/tor...Various improvements could be done to the best-practices tracker (practracker legacy/trac#29221). Here are a few from Nick's last review:
https://github.com/torproject/tor/pull/742#discussion_r262564420
https://github.com/torproject/tor/pull/742#discussion_r262565153
~~https://github.com/torproject/tor/pull/742#discussion_r262566041~~
https://github.com/torproject/tor/pull/742#discussion_r262566501
https://github.com/torproject/tor/pull/742#discussion_r262567882
This could also be a fine starting volunteer project.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29617OOM manger wipes entire DNS cache2020-06-27T13:50:46ZpullsOOM manger wipes entire DNS cacheIn relay.c, function cell_queues_check_size, the OOM manager attempts to clear one tenth of MaxMemInQueues bytes from the DNS cache by calling dns_cache_handle_oom. The function dns_cache_handle_oom, in dns.c, runs in a loop removing cac...In relay.c, function cell_queues_check_size, the OOM manager attempts to clear one tenth of MaxMemInQueues bytes from the DNS cache by calling dns_cache_handle_oom. The function dns_cache_handle_oom, in dns.c, runs in a loop removing cached entries that are now+n*time_inc old, until at least the requested number of bytes have been freed. The first iteration of the loop has n=0, and likely will not remove enough bytes. The second iteration is way too aggressive, because:
```
time_inc += 3600; /* Increase time_inc by 1 hour. */
```
This is guaranteed to wipe the entire DNS cache, because in dns_clip_ttl the maximum time to cache is MAX_DNS_TTL_AT_EXIT, which is set in dns.h to:
```
/** Lowest value for DNS ttl that a server will give. */
#define MIN_DNS_TTL_AT_EXIT (5*60)
/** Highest value for DNS ttl that a server will give. */
#define MAX_DNS_TTL_AT_EXIT (60*60)
```
One possible and reasonable fix would be to instead increment time_inc by MIN_DNS_TTL_AT_EXIT.Tor: 0.3.5.x-finalNick MathewsonNick Mathewson