Tor issueshttps://gitlab.torproject.org/tpo/core/tor/-/issues2023-10-18T14:40:19Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40870Client-side Non-fatal assertion !(smartlist_len(cfx->legs) <= 0) failed in co...2023-10-18T14:40:19ZMike PerryClient-side Non-fatal assertion !(smartlist_len(cfx->legs) <= 0) failed in conflux_pick_first_legWe got a driveby log barf from a cypherpunks:
```
TorProvider: [Tor warning] Bug: Tor 0.4.8.7 (git-bf5e234d): Non-fatal assertion !(smartlist_len(cfx->legs) <= 0) failed in conflux_pick_first_leg at conflux.c:565. (Stack trace not avail...We got a driveby log barf from a cypherpunks:
```
TorProvider: [Tor warning] Bug: Tor 0.4.8.7 (git-bf5e234d): Non-fatal assertion !(smartlist_len(cfx->legs) <= 0) failed in conflux_pick_first_leg at conflux.c:565. (Stack trace not available) (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
TorProvider: [Tor warning] conflux_pick_first_leg: Bug: Matching client sets: (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
TorProvider: [Tor warning] conflux_log_set: Bug: Conflux 6EB994986FCDAA7A: 0 linked, 0 launched. Delivered: 692335; teardown: 0; Current: 0000000000000000, Previous: 0000000000000000 (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
TorProvider: [Tor warning] conflux_pick_first_leg: Bug: Matching server sets: (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
TorProvider: [Tor warning] conflux_log_set: Bug: Conflux 6EB994986FCDAA7A: 0 linked, 0 launched. Delivered: 692335; teardown: 0; Current: 0000000000000000, Previous: 0000000000000000 (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
TorProvider: [Tor warning] conflux_pick_first_leg: Bug: End conflux set dump (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
TorProvider: [Tor warning] circuit_get_package_window: Bug: Conflux has no circuit to send on. Circuit 000001896a32af00 idx 5 marked at line circuitlist.c:1693 (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
TorProvider: [Tor warning] tor_bug_occurred_: Bug: conflux.c:565: conflux_pick_first_leg: Non-fatal assertion !(smartlist_len(cfx->legs) <= 0) failed. (on Tor 0.4.8.7 bf5e234d) TorProvider.sys.mjs:936
```
This looks like a client-side issue. The strange thing is that unlike #40842, this is client-side (Tor Browser, I believe). Additionally, `in_full_teardown` is false, but both the `curr_leg` and the `prev_leg` are NULL, *and* the set was previously used to receive a bunch of data (delivered 692335 cells, as the log says).
The mark for close line is in `circuit_unlink_all_from_channel()`.
What I think might be happening is some kind of shutdown race, where the `curr_leg` is quickly nulled out via `linked_circuit_free()` (which does *not* set `in_full_teardown` -- this could be a bug), but the `prev_leg` is still going through the channel teardown path and getting marked first, and then hitting this situation, without `in_full_teardown` set.
Other than shutdown, I can't think of any other ways that the `curr_leg` could get all the way freed and NULL, without setting `in_full_teardown`. I suppose it is also possible that there were previous warn logs related to this that they omitted.
Unfortunately, the cypherpunk disappeared, and we have no idea if their Tor Browser was having connectivity issues, or if they restarted their Tor, or what else might be a factor.
So we need another instance from a helpful reporter. Ideally one that also has a backtrace (I think this must be windows, because the backtrace is missing).Tor: 0.4.8.x-post-stableMike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40861Look into build failures on modern Debian for CI on main2024-03-20T14:33:46ZAlexander Færøyahf@torproject.orgLook into build failures on modern Debian for CI on main@weasel reported a build issue on i386 with test failures. Have a look at:
```
180923 19:04:59 + weasel: sandbox/openat_filename: [forking] OK
180923 19:05:00 + weasel: sandbox/opendir_dirname: [forking]
180923 19:05:00 + ...@weasel reported a build issue on i386 with test failures. Have a look at:
```
180923 19:04:59 + weasel: sandbox/openat_filename: [forking] OK
180923 19:05:00 + weasel: sandbox/opendir_dirname: [forking]
180923 19:05:00 + weasel: FAIL ../src/test/test_sandbox.c:266: opendir: Operation not permitted [1]
180923 19:05:00 + weasel: [opendir_dirname FAILED]
180923 19:05:00 + weasel: sandbox/chmod_filename: [forking] OK
180923 19:05:00 + weasel: sandbox/chown_filename: [forking] OK
180923 19:05:02 + weasel: hm.
180923 19:05:17 + weasel: https://gitlab.torproject.org/tpo/core/debian/tor/-/jobs/367821
180923 19:05:59 + weasel: https://gitlab.torproject.org/tpo/core/debian/tor/-/jobs/367812
180923 19:06:00 + weasel: https://gitlab.torproject.org/tpo/core/debian/tor/-/jobs/367811
180923 19:06:04 + weasel: https://gitlab.torproject.org/tpo/core/debian/tor/-/jobs/367808
180923 19:06:06 + weasel: they all do that
180923 19:06:35 trinity-1686: maybe related to core/tor!446 ?
180923 19:06:35 -tor:#tor-dev- https://gitlab.torproject.org/tpo/core/tor/-/merge_requests/446 - Fix sandbox on AArch64, RISC-V
180923 19:06:53 + weasel: i386
chat and mess w it
180923 19:09:48 + ahf: weasel: it's only impacting 0.4.9, right?
180923 19:09:56 + ahf: as in, main - not your 0.4.8 builds
180923 19:10:44 + weasel: maybe
180923 19:11:14 + weasel: if that's likely, then I'll push the signed tag and see how those go
180923 19:11:43 + ahf: 0.4.9 is main, so things can break there a bit more frequently than it should on any of the release-* branches
180923 19:12:34 + weasel: pushed the tag, let's see how that pipeline goes
180923 19:12:43 + ahf: oki, let us know how it goes!
180923 19:13:04 + weasel: https://gitlab.torproject.org/tpo/core/debian/tor/-/pipelines/104953 you can know as soon as I know :)
180923 19:13:34 + ahf: i am not going to monitor a build log for a hopeful success here - i am deep into other things in parallel
180923 19:41:39 + ahf: ../src/test/test_bt_cl.c: In function 'crash':
180923 19:41:40 + ahf: ../src/test/test_bt_cl.c:46:24: warning: null pointer dereference [-Wnull-dereference] 46 | *(volatile int *)0 = 0;
180923 19:41:43 + ahf: it's not wrong
180923 20:01:27 + weasel: 0.4.8 did build
180923 20:03:55 + ahf: so 0.4.9 has some build issue on ... all your debian's?
180923 20:06:57 + weasel: just i386 on more modern ones
180923 20:09:45 + ahf: so i386 on modern debian?
180923 20:09:55 + ahf: thank you, weasel
180923 20:09:59 + ahf: creating a ticket
```Tor: 0.4.8.x-post-stableDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40820connection_edge_about_to_close(): Bug: (Harmless.) Edge connection (marked at...2024-03-21T18:27:41Zcomputer_freakconnection_edge_about_to_close(): Bug: (Harmless.) Edge connection (marked at ../src/core/or/circuitlist.c:2713) hasn't sent end yet? (on Tor 0.4.7.13 )```
...
07:55:37.000 [notice] Heartbeat: Tor's uptime is 7 days 18:00 hours, with 71411 circuits open. I've sent 10022.56 GB and received 9786.33 GB. I've received 3866839 connections on IPv4 and 21166 on IPv6. I've made 181900 connectio...```
...
07:55:37.000 [notice] Heartbeat: Tor's uptime is 7 days 18:00 hours, with 71411 circuits open. I've sent 10022.56 GB and received 9786.33 GB. I've received 3866839 connections on IPv4 and 21166 on IPv6. I've made 181900 connections with IPv4 and 56445 with IPv6.
07:55:37.000 [notice] While bootstrapping, fetched this many bytes: 8107 (microdescriptor fetch)
07:55:37.000 [notice] While not bootstrapping, fetched this many bytes: 148937098 (server descriptor fetch); 5760 (server descriptor upload); 12096007 (consensus network-status fetch); 3883848 (microdescriptor fetch)
07:55:37.000 [notice] Circuit handshake stats since last time: 517/517 TAP, 3956544/3973033 NTor.
07:55:37.000 [notice] Since startup we initiated 0 and received 0 v1 connections; initiated 0 and received 0 v2 connections; initiated 0 and received 6320 v3 connections; initiated 3 and received 190964 v4 connections; initiated 71807 and received 3613775 v5 connections.
07:55:37.000 [notice] Heartbeat: DoS mitigation since startup: 1037 circuits killed with too many cells, 32767104 circuits rejected, 401 marked addresses, 1 marked addresses for max queue, 172207 same address concurrent connections rejected, 0 connections rejected, 0 single hop clients refused, 2185090 INTRODUCE2 rejected.
13:37:58.000 [notice] We're low on memory (cell queues total alloc: 10805520 buffer total alloc: 12886016, tor compress total alloc: 667208970 (zlib: 259584, zstd: 666941450, lzma: 0), rendezvous cache total alloc: 109213998). Killing circuits withover-long queues. (This behavior is controlled by MaxMemInQueues.)
13:37:58.000 [notice] Removed 85978221 bytes by killing 62 circuits; 49010 circuits remain alive. Also killed 0 non-linked directory connections. Killed 3 edge connections
13:37:58.000 [warn] connection_edge_about_to_close(): Bug: (Harmless.) Edge connection (marked at ../src/core/or/circuitlist.c:2713) hasn't sent end yet? (on Tor 0.4.7.13 )
13:37:58.000 [warn] tor_bug_occurred_(): Bug: ../src/core/or/connection_edge.c:1065: connection_edge_about_to_close: This line should not have been reached. (Future instances of this warning will be silenced.) (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: Tor 0.4.7.13: Line unexpectedly reached at connection_edge_about_to_close at ../src/core/or/connection_edge.c:1065. Stack trace: (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(log_backtrace_impl+0x57) [0x55ba820720e7] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(tor_bug_occurred_+0x16b) [0x55ba8207d30b] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(connection_exit_about_to_close+0x1d) [0x55ba8211e4fd] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(+0x6b5ed) [0x55ba81ff65ed] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(+0x6bcfb) [0x55ba81ff6cfb] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/lib/x86_64-linux-gnu/libevent-2.1.so.7(+0x23b4f) [0x7fc9eef8db4f] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/lib/x86_64-linux-gnu/libevent-2.1.so.7(event_base_loop+0x52f) [0x7fc9eef8e28f] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(do_main_loop+0x101) [0x55ba81ff86f1] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(tor_run_main+0x1e5) [0x55ba81ff3fc5] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(tor_main+0x49) [0x55ba81ff02d9] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(main+0x19) [0x55ba81fefeb9] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xea) [0x7fc9ee828d0a] (on Tor 0.4.7.13 )
13:37:58.000 [warn] Bug: /usr/bin/tor(_start+0x2a) [0x55ba81feff0a] (on Tor 0.4.7.13 )
13:37:58.000 [warn] connection_edge_about_to_close(): Bug: (Harmless.) Edge connection (marked at ../src/core/or/circuitlist.c:2713) hasn't sent end yet? (on Tor 0.4.7.13 )
13:37:58.000 [warn] connection_edge_about_to_close(): Bug: (Harmless.) Edge connection (marked at ../src/core/or/circuitlist.c:2713) hasn't sent end yet? (on Tor 0.4.7.13 )
13:37:58.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:37:58.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:10.000 [notice] We're low on memory (cell queues total alloc: 15153072 buffer total alloc: 11710464, tor compress total alloc: 664530008 (zlib: 302848, zstd: 664219240, lzma: 0), rendezvous cache total alloc: 109213998). Killing circuits withover-long queues. (This behavior is controlled by MaxMemInQueues.)
13:38:10.000 [notice] Removed 83687644 bytes by killing 17 circuits; 49083 circuits remain alive. Also killed 0 non-linked directory connections. Killed 0 edge connections
13:38:11.000 [notice] We're low on memory (cell queues total alloc: 14386416 buffer total alloc: 12208128, tor compress total alloc: 665847849 (zlib: 259584, zstd: 665580345, lzma: 0), rendezvous cache total alloc: 109213998). Killing circuits withover-long queues. (This behavior is controlled by MaxMemInQueues.)
13:38:11.000 [notice] Removed 88057871 bytes by killing 17 circuits; 49151 circuits remain alive. Also killed 0 non-linked directory connections. Killed 0 edge connections
13:38:11.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:11.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:11.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:11.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:11.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [notice] We're low on memory (cell queues total alloc: 13510992 buffer total alloc: 11796480, tor compress total alloc: 665718009 (zlib: 129792, zstd: 665580345, lzma: 0), rendezvous cache total alloc: 109213998). Killing circuits withover-long queues. (This behavior is controlled by MaxMemInQueues.)
13:38:12.000 [notice] Removed 82329467 bytes by killing 14 circuits; 49226 circuits remain alive. Also killed 0 non-linked directory connections. Killed 0 edge connections
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:12.000 [warn] channel_flush_from_first_active_circuit(): Bug: Found a supposedly active circuit with no cells to send. Trying to recover. (on Tor 0.4.7.13 )
13:38:13.000 [notice] We're low on memory (cell queues total alloc: 14437632 buffer total alloc: 12062720, tor compress total alloc: 664356888 (zlib: 129792, zstd: 664219240, lzma: 0), rendezvous cache total alloc: 109213998). Killing circuits withover-long queues. (This behavior is controlled by MaxMemInQueues.)
13:38:13.000 [notice] Removed 81757690 bytes by killing 15 circuits; 49281 circuits remain alive. Also killed 0 non-linked directory connections. Killed 0 edge connections
13:54:30.000 [notice] Sudden decrease in circuit RTT (11 vs 119683), likely due to clock jump.
13:55:37.000 [notice] Heartbeat: Tor's uptime is 8 days 0:00 hours, with 74885 circuits open. I've sent 10349.14 GB and received 10099.50 GB. I've received 3979142 connections on IPv4 and 21891 on IPv6. I've made 187464 connections with IPv4 and 57946 with IPv6.
13:55:37.000 [notice] While bootstrapping, fetched this many bytes: 8107 (microdescriptor fetch)
13:55:37.000 [notice] While not bootstrapping, fetched this many bytes: 154088181 (server descriptor fetch); 5760 (server descriptor upload); 12495772 (consensus network-status fetch); 4020088 (microdescriptor fetch)
13:55:37.000 [notice] Circuit handshake stats since last time: 581/581 TAP, 2997064/3009741 NTor.
13:55:37.000 [notice] Since startup we initiated 0 and received 0 v1 connections; initiated 0 and received 0 v2 connections; initiated 0 and received 6720 v3 connections; initiated 3 and received 199738 v4 connections; initiated 73618 and received 3714169 v5 connections.
13:55:37.000 [notice] Heartbeat: DoS mitigation since startup: 1051 circuits killed with too many cells, 33500595 circuits rejected, 404 marked addresses, 1 marked addresses for max queue, 183981 same address concurrent connections rejected, 0 connections rejected, 0 single hop clients refused, 2523887 INTRODUCE2 rejected.
...
```https://gitlab.torproject.org/tpo/core/tor/-/issues/40816metrics: New circuit related metrics relay side2024-01-31T07:14:18ZDavid Gouletdgoulet@torproject.orgmetrics: New circuit related metrics relay sideThis is tied to sponsor 112 with https://gitlab.torproject.org/tpo/network-health/team/-/issues/299.
We need to implement these:
- Number of circuit collapsing as in DESTROY coming from the opposite direction.
- Circuit being closed du...This is tied to sponsor 112 with https://gitlab.torproject.org/tpo/network-health/team/-/issues/299.
We need to implement these:
- Number of circuit collapsing as in DESTROY coming from the opposite direction.
- Circuit being closed due to protocol violation.
- Anything around drop cells. A rate, what type, etc...
Basically, keep counters of those events and expose them on the `MetricsPort`.Tor: 0.4.9.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40812socks: Add a new extended error to indicate when client is unable to solve th...2024-01-30T15:30:21ZDavid Gouletdgoulet@torproject.orgsocks: Add a new extended error to indicate when client is unable to solve the PoWWe can use `X'F8'` for this one that would indicate that solving the PoW for a specific onion connection failed.We can use `X'F8'` for this one that would indicate that solving the PoW for a specific onion connection failed.Tor: 0.4.8.x-post-stablehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40806running tor guard/bridge/exit behind layers of nat2023-06-13T20:47:38Zredbearrunning tor guard/bridge/exit behind layers of natThe issue is i can start a node and port is reached by outside but outside ips need uses some intermediate ip to access node. There is some work around ? is possible make something or config on torrc ?The issue is i can start a node and port is reached by outside but outside ips need uses some intermediate ip to access node. There is some work around ? is possible make something or config on torrc ?https://gitlab.torproject.org/tpo/core/tor/-/issues/40807Look for the lib64 directory when using a custom OpenSSL directory2023-06-12T16:28:25ZPier Angelo VendrameLook for the lib64 directory when using a custom OpenSSL directoryFor Tor Browser, we build tor in an old Debian container that has a very old OpenSSL.
So, we also compile OpenSSL and use a custom prefix with `--with-openssl-dir=$openssldir`.
From what I understood (by compiling with `make V=1`), it s...For Tor Browser, we build tor in an old Debian container that has a very old OpenSSL.
So, we also compile OpenSSL and use a custom prefix with `--with-openssl-dir=$openssldir`.
From what I understood (by compiling with `make V=1`), it seems to me that tor's build system tries to use `$openssldir/lib` as a library directory (and IIRC, it falls back to `$openssldir` when it doesn't exist).
However, with OpenSSL 3, the default directory has become `$openssldir/lib64` (at least on Linux amd64).
Linking `$openssldir/lib` to `$openssldir/lib64` seems to solve all the various problems.https://gitlab.torproject.org/tpo/core/tor/-/issues/40805We won't use an L2 guard if it loses Stable, but we don't replace it2023-11-13T19:38:49ZRoger DingledineWe won't use an L2 guard if it loses Stable, but we don't replace itWhile debugging https://gitlab.torproject.org/tpo/core/tor/-/issues/40802, I ran into a fun edge case, where if one of our vanguards-lite loses the Stable flag, we will stop picking it as a middle hop, but in maintain_layer2_guards() we ...While debugging https://gitlab.torproject.org/tpo/core/tor/-/issues/40802, I ran into a fun edge case, where if one of our vanguards-lite loses the Stable flag, we will stop picking it as a middle hop, but in maintain_layer2_guards() we only replace it with a new one when it goes missing from the consensus entirely.
I think that means if all of our vanguards-lite lose the Stable flag (but stay Running) then we'll end up in a sad state where we fail to make any onion-service-related circuits.Roger DingledineRoger Dingledinehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40779Investigate address detection usage in Tor2023-05-15T16:39:11ZAlexander Færøyahf@torproject.orgInvestigate address detection usage in TorTor currently has a number of ways of detecting its own address when being used as relay. This includes:
- netinfo cell
- dirport connections to other relays
- configuration specification
It would be useful for the Arti WG to learn whi...Tor currently has a number of ways of detecting its own address when being used as relay. This includes:
- netinfo cell
- dirport connections to other relays
- configuration specification
It would be useful for the Arti WG to learn which of these methods are actively being used before we start implementing relay support in Arti.
A useful thing to do here is to enumerate the methods we have and extend MetricsPort to store information on where the relay learned its address for so we can make a sensible decision.Tor: 0.4.8.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40782Post-Conflux merge: "All routers are down or won't exit -- choosing a doomed ...2023-05-22T14:22:52ZAlexander Færøyahf@torproject.orgPost-Conflux merge: "All routers are down or won't exit -- choosing a doomed exit at random." log messages@toralf reported the following on #tor-dev IRC the other day:
```
150423 16:53:56 toralf: Th elatest git spams the logs with "All routers are down or won't exit -- choosing a doomed
exit at random...@toralf reported the following on #tor-dev IRC the other day:
```
150423 16:53:56 toralf: Th elatest git spams the logs with "All routers are down or won't exit -- choosing a doomed
exit at random." and "Failed to choose an exit server"
150423 16:56:39 + ahf: toralf: expect git main to be a bit wonky right now since we just landed a pretty huge feature
that is going to need a bit of stabilization
150423 16:56:41 + ahf: mikeperry: ^^
150423 16:57:09 toralf: ahf: thx
150423 17:18:43 + armadev: toralf: does it recover? or does it just do that and be useless as a client?
150423 17:22:08 toralf: I run -git here for 3 servers which do act fine otherwise - from the Grafana logs there is no
significant drop down in throughput nor used sockets, the number of inbound from non-relays
migth however a bit lower than before
150423 17:22:12 toralf: aem
150423 17:22:15 toralf: armadev: ^^
150423 17:27:36 toralf: and at my desktop where I do use Tor as a client only it just sapms the log ful, but it
connects fine to one of my bridges
```
All timestamps are UTC.Tor: 0.4.8.x-freezeMike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40767Investigate high circuit build error rates in simulation2023-04-12T14:46:43Zgabi-250Investigate high circuit build error rates in simulationWe ran some shadow simulations to debug/repro the issue from #40570, and @jnewsome noticed the onion service clients have consistently high [circuit build failure rates](https://gitlab.torproject.org/tpo/core/tor/-/issues/40570#note_2883...We ran some shadow simulations to debug/repro the issue from #40570, and @jnewsome noticed the onion service clients have consistently high [circuit build failure rates](https://gitlab.torproject.org/tpo/core/tor/-/issues/40570#note_2883257).
We should figure out what causes these circuit build failures.https://gitlab.torproject.org/tpo/core/tor/-/issues/40781Tune algorithms for conflux2023-05-25T18:08:32ZMike PerryTune algorithms for confluxI need to run a bunch of shadow sims to tune the conflux algorithms for latency, throughput, and mem usage, and then update the spec for the new algs.
As part of this, we also need to improve the throughput for simulated Hong Kong endpo...I need to run a bunch of shadow sims to tune the conflux algorithms for latency, throughput, and mem usage, and then update the spec for the new algs.
As part of this, we also need to improve the throughput for simulated Hong Kong endpoints. For some reason, they did not improve as much as Germany did. I suspect the problem is that only switching once per cwnd may be handicapping this throughput for higher latency links. We probably need to allow it to switch more often.
When this is done, we also need a changelog entry: https://gitlab.torproject.org/tpo/core/tor/-/issues/40780Tor: 0.4.8.x-freezeMike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40768Decide whether to disable circuit cannibalization entirely2023-04-12T14:47:42Zgabi-250Decide whether to disable circuit cannibalization entirelyThe discussions around #40570 reignited a discussion about circuit cannibalization, and whether the performance improvements it provides (if any) justify the maintenance costs of keeping it around. This ticket is about deciding whether w...The discussions around #40570 reignited a discussion about circuit cannibalization, and whether the performance improvements it provides (if any) justify the maintenance costs of keeping it around. This ticket is about deciding whether we should disable cannibalization in c-tor.https://gitlab.torproject.org/tpo/core/tor/-/issues/40761DDoS mitigation: analysis to understand relay-to-relay connections from non-r...2023-04-12T14:46:14ZbnmDDoS mitigation: analysis to understand relay-to-relay connections from non-relay IPs
We are working on a tor proposal that should help
with protecting non-guard relays from a large fraction
of the DDoS load.
In first tests we have seen a 55% CPU usage decrease
when deploying our proposed mitigations, but we
want to mak...
We are working on a tor proposal that should help
with protecting non-guard relays from a large fraction
of the DDoS load.
In first tests we have seen a 55% CPU usage decrease
when deploying our proposed mitigations, but we
want to make sure that we are not introducing an
over blocking problem. We know about a few
configurations when a relay will use a source IP that is not in
consensus to connect to other relays (OutboundBindAddress, OutboundBindAddressOR)
but we would like to have some actual data about it.
To measure, understand and solve that potential problem and to
back up the proposal with some actual data we would
like to measure the following on our tor relays:
Log when our non-guard tor relays get
an authenticated relay to relay connection to our ORPort
from a source IP that is not in consensus and not in
the exit lists:
```
timestamp relay-fingerprint source-IP
```
If the "and not in the exit lists" part is too hard,
we can take care of that in post-processing of the logs
to filter them out.
We do not care about client to relay connections and do not want to log them.
Would it be possible to provide a patch or branch
that implements that logging on top of main?
It does not have to be in a release and we will run it
only temporarily.
thank you!https://gitlab.torproject.org/tpo/core/tor/-/issues/40747android: fdsan SIGABRT tor_main_configuration_free2023-04-12T14:45:21Zsbsandroid: fdsan SIGABRT tor_main_configuration_free### Summary
When testing tor-0.4.7.13 on Android 13, I experienced a `SIGABRT` in `tor_main_configuration_free` caused by [fdsan](https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md). The reason why fdsan causes an a...### Summary
When testing tor-0.4.7.13 on Android 13, I experienced a `SIGABRT` in `tor_main_configuration_free` caused by [fdsan](https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md). The reason why fdsan causes an abort is that the owning control socket is closed twice. I noticed this crash as part of testing a release candidate of [OONI Probe Android](https://github.com/ooni/probe-android/) where we embed `libtor.a`.
The corresponding OONI Probe issue is: https://github.com/ooni/probe/issues/2405.
### Steps to reproduce
I do not have a very simple procedure to reproduce the issue that does not involve OONI Probe and its build system.
However, the underlying issue is independent of Android. The only reason why Android matters is that the fdsan notices the double close of the same file descriptor and hence triggers a crash (for Android API level >= 30).
Because of this, here are instructions to reproduce the underlying issue using GNU/Linux (I used Ubuntu 22.04):
1. clone tor
2. `git checkout tor-0.4.7.13`
3. `git apply tor.diff` where `tor.diff` is the following patch:
```diff
diff --git a/src/core/mainloop/connection.c b/src/core/mainloop/connection.c
index cf25213cb1..d690de3892 100644
--- a/src/core/mainloop/connection.c
+++ b/src/core/mainloop/connection.c
@@ -149,6 +149,8 @@
#include "core/or/congestion_control_flow.h"
+#include <stdio.h>
+
/**
* On Windows and Linux we cannot reliably bind() a socket to an
* address and port if: 1) There's already a socket bound to wildcard
@@ -949,6 +951,7 @@ connection_free_minimal(connection_t *conn)
if (SOCKET_OK(conn->s)) {
log_debug(LD_NET,"closing fd %d.",(int)conn->s);
+ fprintf(stderr, "SBSDEBUG: connection_free_minimal %lld\n", (long long)conn->s);
tor_close_socket(conn->s);
conn->s = TOR_INVALID_SOCKET;
}
diff --git a/src/feature/api/tor_api.c b/src/feature/api/tor_api.c
index 88e91ebfd5..fb49d92ad7 100644
--- a/src/feature/api/tor_api.c
+++ b/src/feature/api/tor_api.c
@@ -116,6 +116,11 @@ tor_main_configuration_setup_control_socket(tor_main_configuration_t *cfg)
cfg_add_owned_arg(cfg, "__OwningControllerFD");
cfg_add_owned_arg(cfg, buf);
+ fprintf(
+ stderr, "SBSDEBUG: tor_main_configuration_setup_control_socket %lld %lld\n",
+ (long long)fds[0], (long long)fds[1]
+ );
+
cfg->owning_controller_socket = fds[1];
return fds[0];
}
@@ -132,6 +137,10 @@ tor_main_configuration_free(tor_main_configuration_t *cfg)
raw_free(cfg->argv_owned);
}
if (SOCKET_OK(cfg->owning_controller_socket)) {
+ fprintf(
+ stderr, "SBSDEBUG: tor_main_configuration_free %lld\n",
+ (long long)cfg->owning_controller_socket
+ );
raw_closesocket(cfg->owning_controller_socket);
}
raw_free(cfg);
```
4. `./autogen.sh`
5. `./configure --disable-asciidoc`
6. `make`
7. `mkdir tmp`
8. `vi tmp/main.c` making sure it contains the following content:
```C
#include "../src/feature/api/tor_api.h"
#include <stdlib.h>
#include <unistd.h>
int main() {
tor_main_configuration_t *cfg = tor_main_configuration_new();
if (cfg == NULL) {
exit(1);
}
tor_control_socket_t sock = tor_main_configuration_setup_control_socket(cfg);
if (sock == INVALID_TOR_CONTROL_SOCKET) {
exit(2);
}
(void)close(sock); // close immediately (it's async on Android but it should not matter AFAICT)
(void)tor_run_main(cfg);
tor_main_configuration_free(cfg);
}
```
9. `gcc -Wall tmp/main.c -L. -ltor -levent -lcrypto -lssl -lz -lm`
10. `./a.out` which should produce this output:
```
SBSDEBUG: tor_main_configuration_setup_control_socket 4 5
Feb 02 17:18:07.330 [notice] Tor 0.4.7.13 (git-7c1601fb6edd780f) running on Linux with Libevent 2.1.12-stable, OpenSSL 3.0.2, Zlib 1.2.11, Liblzma N/A, Libzstd N/A and Glibc 2.35 as libc.
Feb 02 17:18:07.330 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://support.torproject.org/faq/staying-anonymous/
Feb 02 17:18:07.330 [notice] Configuration file "/usr/local/etc/tor/torrc" not present, using reasonable defaults.
Feb 02 17:18:07.331 [notice] Opening Socks listener on 127.0.0.1:9050
Feb 02 17:18:07.331 [notice] Opened Socks listener connection (ready) on 127.0.0.1:9050
Feb 02 17:18:07.000 [notice] Bootstrapped 0% (starting): Starting
Feb 02 17:18:07.000 [notice] Starting with guard context "default"
Feb 02 17:18:07.000 [notice] Owning controller connection has closed -- exiting now.
SBSDEBUG: connection_free_minimal 5
Feb 02 17:18:07.000 [notice] Catching signal TERM, exiting cleanly.
SBSDEBUG: connection_free_minimal 8
SBSDEBUG: tor_main_configuration_free 5
```
If you analyze the above output, you would see that the file descriptor `5` is closed twice. This output is almost identical to the output that I have seen in the Android logcat (more on that below). Also, I _think_ the way in which I am using the embedding API above (which mirrors our more complex implementation written in Go) is fine; if not, please educate me.
(If you want to reproduce the same problem I experience on Android, I can either explain how to compile and test OONI Probe for Android, or I can try to work on creating a simple Android PoC like the one above.)
### What is the current bug behavior?
We're in an embedding scenario where we eventually call `tor_run_main`, as mentioned in the previous section.
This is the sequence of APIs we call along with my best understanding of what happens inside `tor`:
We create a configuration using `tor_main_configuration_new`.
Calling `tor_main_configuration_setup_control_socket` creates a pair of sockets, returns `fds[0]` to us, and retains `fds[1]` inside `tor_main_configuration_t::owning_controller_socket`.
Calling `tor_run_main` calls (in a way that is not 100% clear to me) `options_act` that passes the `owning_controller_socket` to `control_connection_add_local_fd`. In turn, this function registers the file descriptor `fds[1]` as the control connection.
Eventually we `close` the `fds[0]` that was returned to us, which causes `tor` to stop its libevent loop.
When `tor_run_main` terminates, it calls `tor_cleanup`, which calls `tor_free_all`, which calls `connection_free_all`, which calls `connection_free_minimal` for each connection, including the owning file descriptor `fds[1]`.
After `tor_run_main`, we call `tor_main_configuration_free`. In turn, this function calls `raw_closesocket` on the `owning_controller_socket`, which is hence closed for the second time.
On Android with API level >= 30, the [fdsan](https://android.googlesource.com/platform/bionic/+/master/docs/fdsan.md) sanitizer notices the second close and _sometimes_ (roughly 50%) this fact causes the app to abort.
### What is the expected behavior?
I think tor should duplicate the file descriptor before registering it into the core event loop such that there is a single owner of each of the two duplicates. The `tor_main_configuration_free` function owns one of them and the core event loop owns the other one. This semantics shouldn't cause any issue with the fdsan sanitizer because it's designed to enforce it.
Alternatively, it should probably be documented to use API level < 30 (where the fdsan only warns). Or it should be documented that one should use the proper fdsan API to disable crashing on double close. (I do not remember seeing these warnings when I read how to use the embedding API and a quick `git grep fdsan` or `git grep "API level"` did not return anything, but it's still possible that I overlooked _some_ documentation about this issue.)
### Environment
- Which version of Tor are you using?
tor-0.4.7.13
- Which operating system are you using?
Android 13 (but I also provided a minimal example on GNU/Linux)
- Which installation method did you use?
We cross compile tor for Android using [our cross compilation scripts](https://github.com/ooni/probe-cli/tree/v3.17.0-alpha.1/internal/cmd/buildtool).
We obtain a static set of libraries and a `tor_api.h` that we link as part of building an AAR with [go mobile](https://github.com/golang/mobile).
We use the obtained AAR as a dependency for [OONI Probe Android](https://github.com/ooni/probe-android/).
The code that specifically invokes tor [is written in Go](https://github.com/ooni/probe-android/). The sequence of events in terms of the Tor embedding API is the one I described above in the "what is the current bug behavior?" section.
However, I have also provided a minimal example for GNU/Linux that shows the double-close issue.
### Relevant logs and/or screenshots
The following is an excerpt from the tombstone generated by the crashing app:
```
[notice] Catching signal TERM, exiting cleanly.
fdsan: attempted to close file descriptor 104, expected to be unowned, \
actually owned by unique_fd 0x70b7e5f19c
[...]
ABI: 'arm64'
Timestamp: 2023-02-02 11:39:31.181519664+0100
Cmdline: org.openobservatory.ooniprobe.experimental
pid: 16472, tid: 16593, name: AsyncTask #1 >>> org.openobservatory.ooniprobe.experimental <<<
signal 6 (SIGABRT), code -1 (SI_QUEUE), fault addr --------
[...]
backtrace:
#00 pc 0000000000055c48 .../lib64/bionic/libc.so (fdsan_error(char const*, ...)+556) (...)
#01 pc 0000000000055954 .../lib64/bionic/libc.so (android_fdsan_close_with_tag+732) (...)
#02 pc 00000000000560a8 .../lib64/bionic/libc.so (close+16) (...)
#03 pc 00000000012ad08c [...]/lib/arm64/libgojni.so (tor_main_configuration_free+128)
```
If I apply the patch that above I called `tor.diff` and run OONI Probe on Android, I see this in the logcat:
```
SBSDEBUG: tor_main_configuration_setup_control_socket 94 98 // <- fds[0] and fds[1]
[...]
[notice] Catching signal TERM, exiting cleanly.
SBSDEBUG: connection_free_minimal 141
SBSDEBUG: connection_free_minimal 98 // <- first close of fds[1]
SBSDEBUG: connection_free_minimal 116
SBSDEBUG: connection_free_minimal 152
SBSDEBUG: tor_main_configuration_free 98 // <- second close of fds[1]
```
### Possible fixes
The following patch makes the app work as intended (i.e., no crashes for several runs):
```diff
diff --git a/src/feature/api/tor_api.c b/src/feature/api/tor_api.c
index 88e91ebfd5..2773949264 100644
--- a/src/feature/api/tor_api.c
+++ b/src/feature/api/tor_api.c
@@ -131,9 +131,13 @@ tor_main_configuration_free(tor_main_configuration_t *cfg)
}
raw_free(cfg->argv_owned);
}
+ /* See https://github.com/ooni/probe/issues/2405 to understand
+ why we're not closing the controller socker here. */
+ /*
if (SOCKET_OK(cfg->owning_controller_socket)) {
raw_closesocket(cfg->owning_controller_socket);
}
+ */
raw_free(cfg);
}
```
That said, I think this patch is wrong because it leaks the file descriptor when `tor_run_main` returns prematurely (e.g., when the command line flags are wrong). Because of this, I think the more robust fix would be to duplicate the file descriptor before registering it into the libevent loop, as I explained above.Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40748Change the default of relays per IP address to 42023-05-31T13:11:24ZGeorg KoppenChange the default of relays per IP address to 4Over in #40744 we started allowing 4 relays per IP address via the `AuthDirMaxServersPerAddr` option. I think we should raise the current default, which is 2, accordingly and then update all the specs affected. We might even need a new p...Over in #40744 we started allowing 4 relays per IP address via the `AuthDirMaxServersPerAddr` option. I think we should raise the current default, which is 2, accordingly and then update all the specs affected. We might even need a new proposal for that (including following the whole proposal process)?Tor: 0.4.8.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40168Circuit Build Timeout should ignore abandon circuits, lower min timeout2022-02-28T18:16:02ZMike PerryCircuit Build Timeout should ignore abandon circuits, lower min timeoutWhile working on https://gitlab.torproject.org/tpo/core/tor/-/issues/40157, @asn and I discovered that the Circuit Build Timeout learned by CBT was too high. It is counting failed circuits and circuits that did not complete as if they we...While working on https://gitlab.torproject.org/tpo/core/tor/-/issues/40157, @asn and I discovered that the Circuit Build Timeout learned by CBT was too high. It is counting failed circuits and circuits that did not complete as if they were the longest build-time ("right-censored" in the Pareto estimator). This caused the resulting Pareto curve to be a poor fit. These circuits weren't actually that slow; their relays were just down or unresponsive. We should not count these circuits in our build time stats. Doing so in onionperf testing makes the Pareto curve fit much tighter.
Additionally, there is a #define CBT_DEFAULT_TIMEOUT_MIN_VALUE that makes the minimum timeout value 1.5 seconds. This is way too high -- timeouts on the network range from 80ms to 500ms now. This minimum should be like 25 or 50ms now.
This should be backported to stable clients for a couple reasons -- we want to eventually tune the network-wide circuit build timeout quantile for Sponsor61 experiments, but we can't do that until all clients compute the same circuit build timeout. Additionally, it is abysmal UX to wait for 1.5 seconds on circuits that will never ever build.Sponsor 61 - Making the Tor network faster & more reliable for users in Internet-repressive placesMike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40704relay: Onionskin wait cutoff and MaxOnionQueueDelay in queue should be conse...2022-11-12T09:00:36ZDavid Gouletdgoulet@torproject.orgrelay: Onionskin wait cutoff and MaxOnionQueueDelay in queue should be consensus parametersOur onion queue code, using the CPU thread pool, has this value hardcoded:
```
/** 5 seconds on the onion queue til we just send back a destroy */
#define ONIONQUEUE_WAIT_CUTOFF 5
```
This is used when we add an onionskin to the qu...Our onion queue code, using the CPU thread pool, has this value hardcoded:
```
/** 5 seconds on the onion queue til we just send back a destroy */
#define ONIONQUEUE_WAIT_CUTOFF 5
```
This is used when we add an onionskin to the queue, we will drop any requests that has been longer than that in the queue by sending a `DESTROY`.
There is also a torrc option named `MaxOnionQueueDelay` that behaves a bit differently. If it takes tor more time than that value to process an ntor, reject it.
Both of these should be consensus parameters because they can affect strongly how our relays behave in times of load or attack like at the moment. For instance, under the DoS conditions of the network, it is possible (unproven) that extending `MaxOnionQueueDelay` to a longer wait time could result in less overload from our relays. But, this is a torrc option meaning that if not all 6000 relays update at once, we might have this partitioning problem.
If one group of the network sets that value higher leading to more room to handle onionskins, it means that these relays will have a higher CBT value which would transition circuit creation away from them to more overloaded relays.
If the network all at once change that value, CBT should in theory remains uniform for all path but just that all the sudden, circuit creation fails less but takes more time.Tor: 0.4.8.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40702Single Onion Service Rends become 7 hop after retry2023-02-09T16:22:00ZMike PerrySingle Onion Service Rends become 7 hop after retryIn `retry_service_rendezvous_point()`, if a rend connect fails for a non-anonymous rend, we promote it to a 7-hop slow rend for some reason.
This will impact non-anonymous onions who want performance, especially during the DoS.
David n...In `retry_service_rendezvous_point()`, if a rend connect fails for a non-anonymous rend, we promote it to a 7-hop slow rend for some reason.
This will impact non-anonymous onions who want performance, especially during the DoS.
David notes that this decision to fall back to full anonymous mode in the event of timeout or failure was explicitly written just in case a non-anonymous onion service was also behind a restrictive firewall, and that firewall was the thing that happened to cause a timeout. There is also a comment that explains this, believe it or not. Back then, decision making in C-Tor was a bit more...special.
I bet if we get funders who actually care about single onion performance, they would prefer that their single onions not randomly double in latency on a timeout or failure, just to support the case where some single onion out there might be behind a firewall that they don't know about. Such a funder might suggest that we provide some other option for people behind firewalls to use, instead of this madness.
But I look forward to more research.https://gitlab.torproject.org/tpo/core/tor/-/issues/40217Reproduce Rob's FlashFlood experiment2023-06-08T17:53:42ZMike PerryReproduce Rob's FlashFlood experimentShadow was unable to reproduce the 95th percentile perf degradation for FlashFlood (seen in https://gitlab.torproject.org/tpo/metrics/analysis/-/issues/33076#note_2569011).
I suspect this is due to a Tor bug in something that Shadow is ...Shadow was unable to reproduce the 95th percentile perf degradation for FlashFlood (seen in https://gitlab.torproject.org/tpo/metrics/analysis/-/issues/33076#note_2569011).
I suspect this is due to a Tor bug in something that Shadow is emulating rather than simulating. DNS is in this category. It could be DNS timeouts due to overload.
We should get some fast machines and reproduce the experiment on Live, with more frequent onionperf measurement, so we can get more datapoints of which relays were involved in the 95th percentile slowdown.
Rob has released code.
Tor branch:
https://github.com/robgjansen/tor/tree/research/speedtest/v1-squashed
Python script to run the experiment:
https://gist.github.com/robgjansen/ebd7f8ba019dbef2af4877122281cf3b
Notes from Rob:
- The git log in the Tor branch has some details.
- Several important items are hard-coded in the python "speedtester" script, e.g., the 2nd hop test relay fingerprints and the path a latest consensus file.
- The 2nd hop test relays should set MaxAdvertisedBandwidth to the minimum allowed value so they are reserved as much as possible for the speed testing.
Rob also gave me the Tor Safety Board submission. I will attach that. He also gave me result files, which I will attach to https://gitlab.torproject.org/tpo/metrics/analysis/-/issues/33076 in case we can still figure out the issue from the last run's data.Mike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40479SIGHUP managed proxies when logs reopened2022-04-21T16:37:20ZtwildeSIGHUP managed proxies when logs reopenedWe use SIGHUP to cause Tor to reopen its logs after a rotation, but currently this SIGHUP is not passed along to a managed proxy (ie obfsproxy). The only way to automatically tell obfsproxy to reopen logs without finding its process and...We use SIGHUP to cause Tor to reopen its logs after a rotation, but currently this SIGHUP is not passed along to a managed proxy (ie obfsproxy). The only way to automatically tell obfsproxy to reopen logs without finding its process and sending a SIGHUP manually is to do a full tor restart, which will kill and restart the managed proxy as well. Perhaps all SIGHUPs to Tor should just be passed along to any (unchanged argument) managed proxies?https://gitlab.torproject.org/tpo/core/tor/-/issues/40520module thinko in src/lib/crypt_ops/crypto_rand.c2022-07-07T00:47:11Ztoralfmodule thinko in src/lib/crypt_ops/crypto_rand.c### Summary
I do wonder if
- "modulo 5" should be "modulo 8"
- the bits -> bytes calculation have to made afterwards
- the goal is to calculate the amount of bytes needed for N base32 characters then even the "+7" is wrong
```patch
ind...### Summary
I do wonder if
- "modulo 5" should be "modulo 8"
- the bits -> bytes calculation have to made afterwards
- the goal is to calculate the amount of bytes needed for N base32 characters then even the "+7" is wrong
```patch
index 5bf3a65b3b..90a755537c 100644
--- a/src/lib/crypt_ops/crypto_rand.c
+++ b/src/lib/crypt_ops/crypto_rand.c
@@ -568,9 +568,10 @@ crypto_random_hostname(int min_rand_len, int max_rand_len, const char *prefix,
prefixlen = strlen(prefix);
resultlen = prefixlen + strlen(suffix) + randlen + 16;
- rand_bytes_len = ((randlen*5)+7)/8;
- if (rand_bytes_len % 5)
- rand_bytes_len += 5 - (rand_bytes_len%5);
+ rand_bytes_len = randlen*5;
+ if (rand_bytes_len % 8)
+ rand_bytes_len += 8 - (rand_bytes_len%8);
+ rand_bytes_len /= 8;
rand_bytes = tor_malloc(rand_bytes_len);
crypto_rand(rand_bytes, rand_bytes_len);
```
Or?Tor: 0.4.7.x-stablehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40294Detect the need for and enable '__USE_MINGW_ANSI_STDIO=0' in configure2021-10-19T12:29:39ZAlexander Færøyahf@torproject.orgDetect the need for and enable '__USE_MINGW_ANSI_STDIO=0' in configureCurrently when building Tor on Windows with a modern MinGW-based toolchain, it is necessary to have `CFLAGS='-D__USE_MINGW_ANSI_STDIO=0'"` defined. We should detect this in `configure` and enable it if needed automatically.
In commit d9...Currently when building Tor on Windows with a modern MinGW-based toolchain, it is necessary to have `CFLAGS='-D__USE_MINGW_ANSI_STDIO=0'"` defined. We should detect this in `configure` and enable it if needed automatically.
In commit d9cc2b2928eab045f89d0abf95a9e5c75e290ff8, this was enabled for our AppVeyor builds by explicitly setting the definition in the `CFLAGS` variable.
An example of the error that happens when this is not set:
```
In file included from ./src/lib/crypt_ops/crypto_rsa.h:21,
from ./src/core/or/or.h:31,
from src/core/mainloop/connection.c:58:
src/core/mainloop/connection.c: In function ‘connection_free_minimal’:
src/core/mainloop/connection.c:875:16: error: unknown conversion type character ‘l’ in format [-Werror=format=]
875 | "Freeing orconn at %p, saw channel %p with ID "
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./src/lib/log/log.h:274:43: note: in definition of macro ‘log_info’
274 | log_fn_(LOG_INFO, domain, __FUNCTION__, args, ##__VA_ARGS__)
| ^~~~
src/core/mainloop/connection.c:875:16: error: too many arguments for format [-Werror=format-extra-args]
875 | "Freeing orconn at %p, saw channel %p with ID "
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./src/lib/log/log.h:274:43: note: in definition of macro ‘log_info’
274 | log_fn_(LOG_INFO, domain, __FUNCTION__, args, ##__VA_ARGS__)
| ^~~~
src/core/mainloop/connection.c: In function ‘connection_close_immediate’:
src/core/mainloop/connection.c:1056:21: error: unknown conversion type character ‘l’ in format [-Werror=format=]
1056 | log_info(LD_NET,"fd %d, type %s, state %s, %"TOR_PRIuSZ" bytes on outbuf.",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./src/lib/log/log.h:274:43: note: in definition of macro ‘log_info’
274 | log_fn_(LOG_INFO, domain, __FUNCTION__, args, ##__VA_ARGS__)
| ^~~~
src/core/mainloop/connection.c:1056:21: error: too many arguments for format [-Werror=format-extra-args]
1056 | log_info(LD_NET,"fd %d, type %s, state %s, %"TOR_PRIuSZ" bytes on outbuf.",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./src/lib/log/log.h:274:43: note: in definition of macro ‘log_info’
274 | log_fn_(LOG_INFO, domain, __FUNCTION__, args, ##__VA_ARGS__)
| ^~~~
src/core/mainloop/connection.c: In function ‘connection_connect_sockaddr’:
src/core/mainloop/connection.c:2242:10: error: unknown conversion type character ‘l’ in format [-Werror=format=]
2242 | "Connection to socket %s (sock "TOR_SOCKET_T_FORMAT").",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./src/lib/log/log.h:284:43: note: in definition of macro ‘log_fn’
284 | log_fn_(severity, domain, __FUNCTION__, args, ##__VA_ARGS__)
| ^~~~
src/core/mainloop/connection.c:2242:10: error: too many arguments for format [-Werror=format-extra-args]
2242 | "Connection to socket %s (sock "TOR_SOCKET_T_FORMAT").",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
./src/lib/log/log.h:284:43: note: in definition of macro ‘log_fn’
284 | log_fn_(severity, domain, __FUNCTION__, args, ##__VA_ARGS__)
| ^~~~
src/core/mainloop/connection.c: In function ‘connection_dump_buffer_mem_stats’:
src/core/mainloop/connection.c:5552:6: error: unknown conversion type character ‘l’ in format [-Werror=format=]
5552 | "In buffers for %d connections: %"PRIu64" used/%"PRIu64" allocated",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./src/lib/cc/compat_compiler.h:16,
from ./src/core/or/or.h:26,
from src/core/mainloop/connection.c:58:
/usr/share/mingw-w64/include/inttypes.h:36:18: note: format string is defined here
36 | #define PRIu64 "llu"
| ^
src/core/mainloop/connection.c:5552:6: error: unknown conversion type character ‘l’ in format [-Werror=format=]
5552 | "In buffers for %d connections: %"PRIu64" used/%"PRIu64" allocated",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./src/lib/cc/compat_compiler.h:16,
from ./src/core/or/or.h:26,
from src/core/mainloop/connection.c:58:
/usr/share/mingw-w64/include/inttypes.h:36:18: note: format string is defined here
36 | #define PRIu64 "llu"
| ^
src/core/mainloop/connection.c:5552:6: error: too many arguments for format [-Werror=format-extra-args]
5552 | "In buffers for %d connections: %"PRIu64" used/%"PRIu64" allocated",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
src/core/mainloop/connection.c:5559:9: error: unknown conversion type character ‘l’ in format [-Werror=format=]
5559 | " For %d %s connections: %"PRIu64" used/%"PRIu64" allocated",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./src/lib/cc/compat_compiler.h:16,
from ./src/core/or/or.h:26,
from src/core/mainloop/connection.c:58:
/usr/share/mingw-w64/include/inttypes.h:36:18: note: format string is defined here
36 | #define PRIu64 "llu"
| ^
src/core/mainloop/connection.c:5559:9: error: unknown conversion type character ‘l’ in format [-Werror=format=]
5559 | " For %d %s connections: %"PRIu64" used/%"PRIu64" allocated",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
In file included from ./src/lib/cc/compat_compiler.h:16,
from ./src/core/or/or.h:26,
from src/core/mainloop/connection.c:58:
/usr/share/mingw-w64/include/inttypes.h:36:18: note: format string is defined here
36 | #define PRIu64 "llu"
| ^
src/core/mainloop/connection.c:5559:9: error: too many arguments for format [-Werror=format-extra-args]
5559 | " For %d %s connections: %"PRIu64" used/%"PRIu64" allocated",
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
cc1: all warnings being treated as errors
make[2]: *** [Makefile:12417: src/core/mainloop/connection.o] Error 1
make[2]: Leaving directory '/home/ahf/src/github.com/ahf/tor-win32/src/tor'
make[1]: *** [Makefile:7346: all] Error 2
make[1]: Leaving directory '/home/ahf/src/github.com/ahf/tor-win32/src/tor'
make: *** [Makefile:81: tor] Error 2
```Tor: 0.4.7.x-freezeAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/7457Add client-side log indicator that an obfsbridge works2021-06-18T18:25:22ZGeorge KadianakisAdd client-side log indicator that an obfsbridge worksBridge operators can be reasonably sure that obfsproxy works for them since they see the `Registered server transport` log message.
OTOH, clients can't be sure whether their obfsproxy works or whether they connected to a bridge with obf...Bridge operators can be reasonably sure that obfsproxy works for them since they see the `Registered server transport` log message.
OTOH, clients can't be sure whether their obfsproxy works or whether they connected to a bridge with obfsproxy.
I think we should add PT information to one of the "connected to bridge" log messages.
Specifically, to:
```
log_notice(LD_DIR, "Learned fingerprint %s for bridge %s"
```
or to
```
log_notice(LD_DIR, "new bridge descriptor '%s' (%s): %s"
```
IIUC the latter will appear in the logs everytime we connect to a bridge, even if we knew the bridge fingerprint.Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/8214"getinfo address" should work more consistently soon after startup2021-12-10T09:04:38ZRoger Dingledine"getinfo address" should work more consistently soon after startupTor learns its own address, either by using the interface or by reading the http headers in directory responses.
But if your Tor had enough dir info cached when it came back up, it doesn't make any directory requests for a while. And th...Tor learns its own address, either by using the interface or by reading the http headers in directory responses.
But if your Tor had enough dir info cached when it came back up, it doesn't make any directory requests for a while. And then getinfo address will not have any opinion.
I think the best plan here is to expand the set of things inside Tor that remembers address guesses, to include address guesses we get from netinfo cells.
Wanted by legacy/trac#7348.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/9062Authorities should describe their bwauth version in their votes2020-08-05T20:44:52ZNick MathewsonAuthorities should describe their bwauth version in their votesRight now, there's not a great way to tell which authorities have upgraded their bwauths, which creates trouble as in the case of legacy/trac#8688 . If we have future bwauth software report its version, and we have future Tor authoritie...Right now, there's not a great way to tell which authorities have upgraded their bwauths, which creates trouble as in the case of legacy/trac#8688 . If we have future bwauth software report its version, and we have future Tor authorities check that version and report it in their networkstatus votes, then we'll not stumble into that situation again.Tor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/15380log messages are doubled and unclear2021-07-09T17:22:51Ztoralflog messages are doubled and unclearPlayed today with accounting and read in notice log :
```
Mar 19 18:49:51.000 [warn] You have set AccountingMax to use hibernation. You have also chosen a low DirPort or OrPort. This combination can make Tor stop working when it tries to...Played today with accounting and read in notice log :
```
Mar 19 18:49:51.000 [warn] You have set AccountingMax to use hibernation. You have also chosen a low DirPort or OrPort. This combination can make Tor stop working when it tries to re-attach the port after a period of hibernation. Please choose a different port or turn off hibernation unless you know this combination will work on your platform.
Mar 19 18:49:51.000 [warn] You have set AccountingMax to use hibernation. You have also chosen a low DirPort or OrPort. This combination can make Tor stop working when it tries to re-attach the port after a period of hibernation. Please choose a different port or turn off hibernation unless you know this combination will work on your platform.
Mar 19 18:49:51.000 [notice] Tor 0.2.6.5-rc (git-e0b77cd3194d705f) opening log file.
Mar 19 18:49:51.000 [warn] Failed to unlink /var/lib/tor/data/bw_accounting: No such file or directory
Mar 19 18:49:51.000 [notice] Configured hibernation. This interval begins at 2015-03-01 05:00:00 and ends at 2015-04-01 05:00:00. We have no prior estimate for bandwidth, so we will start out awake and hibernate when we exhaust our quota.
Mar 19 18:49:51.000 [notice] Not advertising DirPort (Reason: AccountingMax enabled)
```
Beside the doubling /me wonders why tor *warns* about DirPort (or ORPort) - but then *noticed* that it will ignore DirPort (anyway).Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40082Add nonfatal_once assertion about is_canonical assertion in `circuit_deliver_...2020-11-12T15:55:57ZNick MathewsonAdd nonfatal_once assertion about is_canonical assertion in `circuit_deliver_create_cell()`Working on #13753: it seems the best way to do that is to add a check for canonicity in circuit_deliver_create_cell().Working on #13753: it seems the best way to do that is to add a check for canonicity in circuit_deliver_create_cell().Tor: 0.4.5.x-stableNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40172Build failure on macOS on master2021-07-09T17:30:28ZMatthew FinkelBuild failure on macOS on masterIn tpo/applications/tor-browser-build#40139 we documented that Tor Browser Nightly build for macOS is failing when linking tor using `master`. This began failing on 24 or 25 October. I haven't bisected which tor commit (or build change) ...In tpo/applications/tor-browser-build#40139 we documented that Tor Browser Nightly build for macOS is failing when linking tor using `master`. This began failing on 24 or 25 October. I haven't bisected which tor commit (or build change) is causing this. Attached is the latest build log when building commit df1637600469 (tip of master, at build time).
[tor-osx-x86_64.log](/uploads/f83b8ec1bc057c5cd7f8774936b35aa5/tor-osx-x86_64.log)Tor: 0.4.5.x-stableNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40198Ensure that Tor builds on MacOS for ARM2022-11-21T02:25:11ZAlexander Færøyahf@torproject.orgEnsure that Tor builds on MacOS for ARMWith the arrival of the new MacOS release for ARM, we should make sure that Tor builds for the MacOS ARM target. It should be possible to cross-compile Tor from MacOS on x86-64 to ARM.
Best way to get this running right now is via the b...With the arrival of the new MacOS release for ARM, we should make sure that Tor builds for the MacOS ARM target. It should be possible to cross-compile Tor from MacOS on x86-64 to ARM.
Best way to get this running right now is via the build-system of Tor.framework.
Benjamin from the Guardian Project says that Tor.framework currently does not work for the cross-compile target for MacOS ARM.Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/29084Ensure circuit padding RTT estimate handes var cells/wide creates2023-06-15T10:31:27ZGeorge KadianakisEnsure circuit padding RTT estimate handes var cells/wide createsThe use_rtt_estimate field in the circuit padding machines lets machines offset the inter-packet delays by a middle-node estimated RTT value of packets that go from the middle to the exit/website.
We abort this measurement if we get two...The use_rtt_estimate field in the circuit padding machines lets machines offset the inter-packet delays by a middle-node estimated RTT value of packets that go from the middle to the exit/website.
We abort this measurement if we get two or more cells back-to-back in either direction, as this indicates that the half-duplex request/response circuit setup and RELAY_BEGIN sequence has finished.
However, if we switch to a multi-cell circuit handshake, then we will need to take that into account when measuring RTT.
If RELAY_EARLY is used only for the first cell of a multi-cell EXTEND2 payload,
then we can just count time between RELAY_EARLIES. But the proposal currently says MAY, but not MUST for this behavior.Mike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29220Update review guidelines to list best practices2021-07-22T16:19:43ZNick MathewsonUpdate review guidelines to list best practicesWhen we come up with some target guidelines in legacy/trac#29219 , we should make sure that our coding standards document them.When we come up with some target guidelines in legacy/trac#29219 , we should make sure that our coding standards document them.https://gitlab.torproject.org/tpo/core/tor/-/issues/29245Tor 0.4 eventually hits "Delaying directory fetches: No running bridges" afte...2023-08-01T23:52:42ZTracTor 0.4 eventually hits "Delaying directory fetches: No running bridges" after some period of inactivity with bridges```
Tor NOTICE: Delaying directory fetches: No running bridges
Tor NOTICE: Application request when we haven't received a consensus with exits. Optimistically trying known bridges again.
Tor NOTICE: Delaying directory fetches: No runni...```
Tor NOTICE: Delaying directory fetches: No running bridges
Tor NOTICE: Application request when we haven't received a consensus with exits. Optimistically trying known bridges again.
Tor NOTICE: Delaying directory fetches: No running bridges
Tor NOTICE: Application request when we haven't received a consensus with exits. Optimistically trying known bridges again.
Tor NOTICE: Delaying directory fetches: No running bridges
Tor NOTICE: Application request when we haven't received a consensus with exits. Optimistically trying known bridges again.
```
Tested on latest Tor Browser alpha with snowflake bridge.
**Trac**:
**Username**: ArmalsLoveArmalsLifeTor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/30797Stop shipping an abandoned systemd script?2021-09-16T14:23:38ZRoger DingledineStop shipping an abandoned systemd script?legacy/trac#19759, legacy/trac#19761, legacy/trac#19762 are bugs filed on the systemd script that we ship in "contrib/dist/tor.service.in".
The bugs haven't been looked at in years, and it looks like nobody on our side is paying attenti...legacy/trac#19759, legacy/trac#19761, legacy/trac#19762 are bugs filed on the systemd script that we ship in "contrib/dist/tor.service.in".
The bugs haven't been looked at in years, and it looks like nobody on our side is paying attention to this systemd script. That sounds to me like we would consider people foolish if they tried to use it.
Should we confirm that somebody, somewhere else on the internet, has a better systemd script than we do, and then remove ours?Tor: 0.4.5.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/32349hs-v2: Intro point circuit TIMEOUT failure is not reported2021-06-23T17:23:06ZDavid Gouletdgoulet@torproject.orghs-v2: Intro point circuit TIMEOUT failure is not reportedThis was found while I was working on legacy/trac#32020.
For v2, we report a TIMEOUT circuit failure within `circuit_about_to_free()`. The following code is the snippet on how we check if the circuit timed out:
```
int reason = circ->m...This was found while I was working on legacy/trac#32020.
For v2, we report a TIMEOUT circuit failure within `circuit_about_to_free()`. The following code is the snippet on how we check if the circuit timed out:
```
int reason = circ->marked_for_close_reason;
int timed_out = (reason == END_CIRC_REASON_TIMEOUT);
```
However, in `circuit_mark_for_close_()`, if the circuit is an origin one, which is the case for all HS client circuit, the `marked_for_close_reason` is set to `END_CIRC_REASON_NONE` so we don't send back that reason back within the destroy cell.
The fix is that we should be looking at `marked_for_close_orig_reason` instead.
We need to backport this.Tor: 0.4.5.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/32697Require supported relay protocol versions based on 0.3.5.72020-10-22T15:48:21ZteorRequire supported relay protocol versions based on 0.3.5.7Like legacy/trac#32696, we should make the 0.3.5.7 protocol versions required for relays some time between April and December 2020.
See:
https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-removal-policy.txt#n36Like legacy/trac#32696, we should make the 0.3.5.7 protocol versions required for relays some time between April and December 2020.
See:
https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-removal-policy.txt#n36Tor: 0.4.5.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/32698Require client protocol versions based on 0.3.5.72020-10-22T15:48:27ZteorRequire client protocol versions based on 0.3.5.7Assuming legacy/trac#32696 is deployed between March and June 2020, we should make the 0.3.5.7 protocol versions required for clients some time in 2021.
See:
https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-removal-...Assuming legacy/trac#32696 is deployed between March and June 2020, we should make the 0.3.5.7 protocol versions required for clients some time in 2021.
See:
https://gitweb.torproject.org/torspec.git/tree/proposals/303-protover-removal-policy.txt#n49Tor: 0.4.5.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/33262Prop 313: 3. Write a Script that Counts IPv6 Relays in the Consensus2020-07-02T19:46:43ZteorProp 313: 3. Write a Script that Counts IPv6 Relays in the ConsensusWe want to write a script that generates statistics for relays that:
1. have an IPv6 ORPort,
2. support IPv6 clients,
3. support IPv6 reachability checks, and
4. support IPv6 reachability checks, and IPv6 clients.
The first two ...We want to write a script that generates statistics for relays that:
1. have an IPv6 ORPort,
2. support IPv6 clients,
3. support IPv6 reachability checks, and
4. support IPv6 reachability checks, and IPv6 clients.
The first two statistics have no dependencies. The last two statistics depend on the "Relay=3" subprotocol in legacy/trac#33226.
The script should calculate:
* the number of relays, and
* the consensus weight fraction of relays.
In order to provide easy access to these statistics, we propose
that the script should:
* download a consensus (or read an existing consensus), and
* calculate and report these statistics.
We could write this script using Python 3 and Stem:
https://stem.torproject.org
The following consensus weight fractions should divide by the total
consensus weight:
* have an IPv6 ORPort (all relays have an IPv4 ORPort), and
* support IPv6 reachability checks (all relays support IPv4 reachability).
The following consensus weight fractions should divide by the
"usable Guard" consensus weight:
* support IPv6 clients, and
* support IPv6 reachability checks and IPv6 clients.
"Usable Guards" have the Guard flag, but do not have the Exit flag. If the
Guard also has the BadExit flag, the Exit flag should be ignored.
The script should check that Wgd is 0. If it is not, the script
should log a warning about the accuracy of the "Usable Guard" statistics.
See proposal 313, section 3:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n82Sponsor 55: Improving the Tor network’s IPv6 supportNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/33268Prop 313: 8.1. Test IPv6 Relay Consensus Counts on the Tor Network2020-07-02T19:47:19ZteorProp 313: 8.1. Test IPv6 Relay Consensus Counts on the Tor NetworkThis task depends on legacy/trac#33262.
Test the IPv6 relay count script in proposal 313, using the public tor network.
There isn't really much to test here. There are only 4 calculations. We've done counts like this before.
See propo...This task depends on legacy/trac#33262.
Test the IPv6 relay count script in proposal 313, using the public tor network.
There isn't really much to test here. There are only 4 calculations. We've done counts like this before.
See proposal 313, section 8.1, public tor network tests part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n337Sponsor 55: Improving the Tor network’s IPv6 supportNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/33272Prop 313: 8.2. Test IPv6 Stats on the Tor Network2020-08-07T13:17:17ZteorProp 313: 8.2. Test IPv6 Stats on the Tor NetworkWe want to test the IPv6 connection and bandwidth statistics on the public network, with a small number of relays and bridges.
See proposal 313, section 8.2, public tor network tests part:
https://gitweb.torproject.org/torspec.git/tree/...We want to test the IPv6 connection and bandwidth statistics on the public network, with a small number of relays and bridges.
See proposal 313, section 8.2, public tor network tests part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n355Sponsor 55: Improving the Tor network’s IPv6 supportNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/33253Prop 312: 5.1. Ask Relay Operators to Test IPv6 Addresses Discovery2020-07-22T19:55:48ZteorProp 312: 5.1. Ask Relay Operators to Test IPv6 Addresses DiscoverySee legacy/trac#33252 for advice on testing stages.
Write an email to the tor-relays list, asking relay operators to help test the proposal 312 IPv4 and IPv6 Address Discovery changes.
Once these changes are merged, volunteer relay and...See legacy/trac#33252 for advice on testing stages.
Write an email to the tor-relays list, asking relay operators to help test the proposal 312 IPv4 and IPv6 Address Discovery changes.
Once these changes are merged, volunteer relay and bridge operators will be able to test them by:
* compiling from source,
* running nightly builds, or
* running alpha releases.
See proposal 312, section 5.1, relay operator test part:
https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv6-addr.txt#n1461Tor: 0.4.5.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/33352Warning: RELAY_COMMAND_INTRODUCE_ACK on padding circuit2022-11-19T20:47:42ZteorWarning: RELAY_COMMAND_INTRODUCE_ACK on padding circuitWhen I run the new mixed+hs-v23-ipv6 chutney network from legacy/trac#33333, I see the following warning:
```
Ignored cell (40) that arrived in padding circuit 32.
```
I am running Tor 3.5.8 and 0.4.4.0-alpha (a6509cb867).
This issue a...When I run the new mixed+hs-v23-ipv6 chutney network from legacy/trac#33333, I see the following warning:
```
Ignored cell (40) that arrived in padding circuit 32.
```
I am running Tor 3.5.8 and 0.4.4.0-alpha (a6509cb867).
This issue appears to be timing-dependent: I saw it on one run, but I did not see it on the next one.Tor: 0.4.4.x-finalGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/core/tor/-/issues/5304Obfsproxy should respect OutboundBindAddress in torrc2020-10-15T16:30:29ZTracObfsproxy should respect OutboundBindAddress in torrcRather it just binds to * (any IP).
Tested with latest git obfsproxy and Tor 0.2.3.12-alpha.
**Trac**:
**Username**: korobkovRather it just binds to * (any IP).
Tested with latest git obfsproxy and Tor 0.2.3.12-alpha.
**Trac**:
**Username**: korobkovTor: 0.4.5.x-freezeDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/33629Use stale bot to close old pull requests2021-02-05T21:03:36ZteorUse stale bot to close old pull requestsShould we automatically close stale pull request on GitHub?
Some GitHub users won't contribute to a project with lots of open pull requests, because they think it's been abandoned. (Or they think we're bad at merging PRs.)
But lots of ...Should we automatically close stale pull request on GitHub?
Some GitHub users won't contribute to a project with lots of open pull requests, because they think it's been abandoned. (Or they think we're bad at merging PRs.)
But lots of our PRs get left open because we've squashed and merged the PR, and GitHub doesn't recognise the squashed commits, so GitHub doesn't auto-close the PR.
Closing a PR:
* hides it from the default list of PRs
* hides its CI results in the PR itself (but they still appear in the PR list)
Anyone can reopen a closed PR. Re-opening a PR re-runs CI (which is usually what we want for old PRs).
I suggest we use probot stale:
https://probot.github.io/apps/stale/
With the following options:
* daysUntilStale: 60
* some backport PRs are older than 60 days, but their CI should have been checked when they were reviewed
* we don't merge very many stale PRs to master
* any period from 30 days (most reviews completed) to 6 months (release cycle) would probably work for us
* daysUntilClose: 7
* exemptLabels: pinned
* staleLabel: stale-closed
* markComment: (like the suggested one, but mention the "pinned" label)
* closeComment: false
On the following network team repositories:
* tor
* ~400 older than 2 months
* ~270 older than 6 months
* torspec
* 8 older than 2 months
* 6 older than 6 months
* chutney
* 1 older than 6 months, needs to be pinned?
* fallback-scripts
And the top-level GitHub config repository:
* .githubhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40185relay with 0.4.5.1-alpha-dev (git-dd119b277b66c1a7) ignores -INT and -USR12021-10-22T14:17:38Ztoralfrelay with 0.4.5.1-alpha-dev (git-dd119b277b66c1a7) ignores -INT and -USR1It happened here 2x in a row that a relay here could only be terminated (-HUP worked all the time for what it is worth).
OS is a hardened Linux.
FWIW there are 2 relays running at the same hardware under a hardened Gentoo Linux.
The one...It happened here 2x in a row that a relay here could only be terminated (-HUP worked all the time for what it is worth).
OS is a hardened Linux.
FWIW there are 2 relays running at the same hardware under a hardened Gentoo Linux.
The one which is listing at default ports (80 and 443) behaves fine, the other (9001 and 9030 and being a FallbackDir too) however is affected by this issue.
Update: It is independent from being a FallbackDir or not, with bc6c37b202e907ed now the other relay refused to restart - and today -TERM was ignored too at 1 relay.
Nowadays a _kill -HUP_ is ignored after an uptime of only few hours.
I use this cronjob for now to keep both relays aware of signals
```
@hourly (/sbin/rc-service tor reload; /sbin/rc-service tor2 reload) 1>/dev/null
```Tor: 0.4.5.x-post-stableAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/34086HSv3: Bug Non-fatal assertion in hs_client.c:776: client_rendezvous_circ_has_...2021-06-23T17:22:07Zs7rHSv3: Bug Non-fatal assertion in hs_client.c:776: client_rendezvous_circ_has_openedClient side HSv3 non fatal bug. It occurs like between ~50 - 70 times in a total of 100.000 built rendezvous circuits:
```
Apr 03 14:53:04.000 [warn] tor_bug_occurred_(): Bug: ../src/feature/hs/hs_client.c:776: client_rendezvous_circ_ha...Client side HSv3 non fatal bug. It occurs like between ~50 - 70 times in a total of 100.000 built rendezvous circuits:
```
Apr 03 14:53:04.000 [warn] tor_bug_occurred_(): Bug: ../src/feature/hs/hs_client.c:776: client_rendezvous_circ_has_opened: Non-fatal assertion !(!node_supports_v3_rendezvous_point(rp_node)) failed. (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: Tor 0.4.4.0-alpha-dev: Non-fatal assertion !(!node_supports_v3_rendezvous_point(rp_node)) failed in client_rendezvous_circ_has_opened at ../src/feature/hs/hs_client.c:776. Stack trace: (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(log_backtrace_impl+0x56) [0x55d874ac7ee6] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(tor_bug_occurred_+0x16c) [0x55d874ac30ec] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(circuit_has_opened+0x80) [0x55d874947810] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(circuit_send_next_onion_skin+0x2b8) [0x55d874930d38] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(+0xbe4ba) [0x55d8749664ba] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(+0xbf1c5) [0x55d8749671c5] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(circuit_receive_relay_cell+0x2b4) [0x55d874968a44] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(command_process_cell+0x2c8) [0x55d87494b788] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(channel_tls_handle_cell+0x4eb) [0x55d87492af0b] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(+0xac5ca) [0x55d8749545ca] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(connection_handle_read+0xa92) [0x55d8749183a2] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(+0x75609) [0x55d87491d609] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: /lib/x86_64-linux-gnu/libevent-2.1.so.6(+0x229ba) [0x7f1984b569ba] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: /lib/x86_64-linux-gnu/libevent-2.1.so.6(event_base_loop+0x5a7) [0x7f1984b57537] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(do_main_loop+0xff) [0x55d87491e88f] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(tor_run_main+0x10b5) [0x55d87490bbc5] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(tor_main+0x3a) [0x55d8749092ca] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(main+0x19) [0x55d874908e89] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f198443809b] (on Tor 0.4.4.0-alpha-dev )
Apr 03 14:53:04.000 [warn] Bug: tor(_start+0x2a) [0x55d874908eda] (on Tor 0.4.4.0-alpha-dev )
```Tor: 0.4.4.x-finalGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/core/tor/-/issues/33226Prop 311: 5. Declare Support for Subprotocol Version "Relay=3"2020-06-27T13:48:19ZteorProp 311: 5. Declare Support for Subprotocol Version "Relay=3"This ticket depends on relay IPv6 extends in legacy/trac#33220.
We reserve Tor subprotocol "Relay=3" for tor versions where:
* relays may perform IPv6 extends, and
* bridges might not perform IPv6 extends,
as described in this propo...This ticket depends on relay IPv6 extends in legacy/trac#33220.
We reserve Tor subprotocol "Relay=3" for tor versions where:
* relays may perform IPv6 extends, and
* bridges might not perform IPv6 extends,
as described in this proposal.
See proposal 311, section 5:
https://gitweb.torproject.org/torspec.git/tree/proposals/311-relay-ipv6-reachability.txt#n601Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/5940Figure out own IPv6 address2020-06-27T14:06:19ZLinus Nordberglinus@torproject.orgFigure out own IPv6 addressA relay should be able to figure out if the system has got an IPv6
address configured, just like what's done for IPv4.
From legacy/trac#5146:
Should we turn resolve_my_address() into resolve_my_addresses() and
teach it about IPv6? ...A relay should be able to figure out if the system has got an IPv6
address configured, just like what's done for IPv4.
From legacy/trac#5146:
Should we turn resolve_my_address() into resolve_my_addresses() and
teach it about IPv6? get_interface_address6() used here needs some
work for legacy/trac#4806 too.
A few thoughts:
- resolve_my_address() looks at options->Address. What should
'Address' mean now that a relay doesn't have one single address any
more?
- get_interface_address() says "This address should only be used in
checking whether our address has changed" but is actually used by
resolve_my_address() in the case where we fail to resolve our
hostname. Does get_interface_address6() need more work or should we
just add a comment to where we use it in a non-recommended way?Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/33527Write walking onions specification2020-06-27T13:48:06ZGabagaba@torproject.orgWrite walking onions specificationWalking onions is a set of protocols improving scalability for the Tor network by enabling constant-size scaling of the information each client must download. Walking Onions will allow us to remove nearly all directory overhead from the ...Walking onions is a set of protocols improving scalability for the Tor network by enabling constant-size scaling of the information each client must download. Walking Onions will allow us to remove nearly all directory overhead from the Tor protocol, enabling Tor to scale to many more clients and relays, with no reduction in security.
The proposal is at https://gitweb.torproject.org/torspec.git/tree/proposals/300-walking-onions.txt
This ticket is a complete, byte-level specification of the Walking Onions design, in sufficient detail to permit independent implementations of Walking Onions to interoperate. This will include a description of all new directory formats, all new wire protocols, all new client and relay behaviors, and all backward compatibility mechanisms.
Activities:
* Write an initial draft of specification, identifying unknowns and options in the design.
* Distribute draft to tor-dev mailing list and to researchers for comment.
* Take decisions on all unknowns and options; if uncertainty remains.
* Write or locate reference-implementations for any primitive operations not already used by Tor.Write reference implementations for all novel encodings/decodings.Tor: 0.4.4.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/33703Improving HS availability under DoS (master ticket)2022-10-11T23:39:34ZGeorge KadianakisImproving HS availability under DoS (master ticket)This is the master ticket for organizing work under this topic.This is the master ticket for organizing work under this topic.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40121gitlab ci is now out-of-sync between master and maint-0.3.52020-09-22T13:23:44ZNick Mathewsongitlab ci is now out-of-sync between master and maint-0.3.5There are big comments at the head of `.gitlab-ci.yml` and `ci-driver.sh` that say to keep them in sync on all our branches. But they aren't in sync any more.
@dgoulet, can you have a look? I think it's your commits for tracing that b...There are big comments at the head of `.gitlab-ci.yml` and `ci-driver.sh` that say to keep them in sync on all our branches. But they aren't in sync any more.
@dgoulet, can you have a look? I think it's your commits for tracing that broke this property.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/33843Write detailed priority queue scheduler design on the proposal2022-10-11T23:39:34ZGeorge KadianakisWrite detailed priority queue scheduler design on the proposalTor: unspecifiedDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/33844Do next iteration of HS-DoS proposal by folding in comments from dgoulet/mike2022-10-11T23:39:35ZGeorge KadianakisDo next iteration of HS-DoS proposal by folding in comments from dgoulet/mikeThis is just a ticket to track progress: making the next iteration of the proposal based on dgoulet/mike comments is the next step here.This is just a ticket to track progress: making the next iteration of the proposal based on dgoulet/mike comments is the next step here.https://gitlab.torproject.org/tpo/core/tor/-/issues/33899Allow IPv6 addresses to be canonical2020-06-27T13:47:53ZteorAllow IPv6 addresses to be canonicalIn legacy/trac#17604, we allowed IPv6 connections to set is_canonical_to_peer. But we did not let them set is_canonical.
So channel_tls_process_netinfo_cell() handles IPv6 addresses, but connection_or_check_canonicity() does not.
This ...In legacy/trac#17604, we allowed IPv6 connections to set is_canonical_to_peer. But we did not let them set is_canonical.
So channel_tls_process_netinfo_cell() handles IPv6 addresses, but connection_or_check_canonicity() does not.
This inconsistency could be one cause of the logging issues in legacy/trac#24841.
We should allow IPv6 connections to be canonical. We should also do some cleanup of the related connection address logging in legacy/trac#33898.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/34412Additonal unit tests for functions related to ipv6 self-test2020-10-22T15:12:33ZNick MathewsonAdditonal unit tests for functions related to ipv6 self-testSee legacy/trac#33222 and legacy/trac#34200 for lists of functions that did not get unit tests as part of the legacy/trac#34200 merge.See legacy/trac#33222 and legacy/trac#34200 for lists of functions that did not get unit tests as part of the legacy/trac#34200 merge.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/21789Use cached address values more often2020-08-17T14:58:14ZteorUse cached address values more oftenarma said in legacy/trac#20423:
Once we get this one in, we might in 0.3.0 consider changing many more of these calls to cache_only=1. In fact, if we leave check_descriptor_ipaddress_changed() calling resolve_my_address() once a minute,...arma said in legacy/trac#20423:
Once we get this one in, we might in 0.3.0 consider changing many more of these calls to cache_only=1. In fact, if we leave check_descriptor_ipaddress_changed() calling resolve_my_address() once a minute, maybe we're all set and the whole of router_pick_published_address() can just look at the cached values? I didn't want to make an invasive change like that in 0.2.9 though, since there are probably edge cases we care about, e.g. where getinfo_helper_misc() calls router_pick_published_address() and we're not a relay so we've never called resolve_my_address() before that very moment.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/26299Reproducible Tor releases2021-10-29T13:21:31ZAlexander Færøyahf@torproject.orgReproducible Tor releasesIt might be useful to have that the generation of the Tor release tarballs are reproducible by multiple people from the Git source tree.
This could allow us to do things like:
- Having multiple people sign a release (since they are able...It might be useful to have that the generation of the Tor release tarballs are reproducible by multiple people from the Git source tree.
This could allow us to do things like:
- Having multiple people sign a release (since they are able to generate the release themselves and sign it).Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40310CI: Somehow, test_switch_id.sh is failing on 0352021-10-22T13:26:01ZDavid Gouletdgoulet@torproject.orgCI: Somehow, test_switch_id.sh is failing on 035See: https://gitlab.torproject.org/dgoulet/tor/-/pipelines/3132See: https://gitlab.torproject.org/dgoulet/tor/-/pipelines/3132Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40140Directory authorities should enable ExtendByEd25519ID2020-10-19T21:44:30ZNick MathewsonDirectory authorities should enable ExtendByEd25519IDThis option has been supported since 0.3.0; it is no longer a fingerprinting risk, and should be enabled for everybody.
@tpo/core Any objections, or should I ask dir-auths@ to do this? Do you think we need to do more testing first?This option has been supported since 0.3.0; it is no longer a fingerprinting risk, and should be enabled for everybody.
@tpo/core Any objections, or should I ask dir-auths@ to do this? Do you think we need to do more testing first?Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40277Circuit Padding attack scenario through circpad_global_max_padding_pct2023-06-09T13:26:55ZGhost UserCircuit Padding attack scenario through circpad_global_max_padding_pctHello *,
I am from [RadicallyOpenSecurity](https://www.radicallyopensecurity.com/) and currently performing a short review of the [Padding Machines for Tor](https://nlnet.nl/project/TorPaddingMachines/) NGI Zero PET project.
During dis...Hello *,
I am from [RadicallyOpenSecurity](https://www.radicallyopensecurity.com/) and currently performing a short review of the [Padding Machines for Tor](https://nlnet.nl/project/TorPaddingMachines/) NGI Zero PET project.
During discussion with the padding machine author Tobias Pulls (@pulls), I noticed a general attack scenario that could be of relevance for the padding framework implementation.
The general idea of the attack is that a malicious Tor client can, under some circumstances, repeatedly force the circuit padding logic of middle relays to reply with more padding bytes than non-padding bytes. After some time of sustained attacks, the target middle relay will develop a statistical circuit padding overhead percentage which is over the the network-wide `circpad_global_max_padding_pct` limit (see [section 3.5](https://gitweb.torproject.org/tor.git/tree/doc/HACKING/CircuitPaddingDevelopment.md?id=33380f6b2770a3c4d9e2a9cc8a4b18c71a40571b#n614) of the documentation), at which point it will stop sending padding on any of its circuits.
Once the attack stops, the ratio between sent padding and sent non-padding of the relay goes back to normal over time while the relay serves non-padded cell data to legitimate clients, so it reverts automatically to a safe state again where it has working padding machines.
However, there are several notable details:
* To our knowledge, the currently rolled out padding machines can not be used to trigger this problem. This is likely only a problem for future machines with high volume that defend against website fingerprinting attacks.
* Due to the event-based nature of client padding machines, the suppression of padding generation at the middle relay means that the `CIRCPAD_EVENT_PADDING_RECV` transition on the client is never taken. This results in the client not sending any padding, or performing only a very limited subset of the regular padding behavior (depending on the machine definition). This amplifies the practical impact of the missing padding from the relay.
* Since the relay behavior change affects all of its circuits at once and may happen several times over a short time-span, the resulting traffic anomalies may be a fingerprinting opportunity for adversaries that are performing traffic analysis on the network.
* According to @pulls, the consequences of this attack on the relay server would only be logged at `info` level and may go unnoticed by relay operators.
* There is a threshold mechanism at the individual circuit machine level through the `relay->max_padding_percent` setting. At first glance, this appears to mitigate the described attack scenario. However, note that this limit is not applied to the first `relay->allowed_padding_count` number of cells, so an attacker can circumvent this limitation by repeatedly triggering new padding machines. Since it appears to be beneficial for padding machines to have this initial level of freedom over padding for their effectiveness against fingerprinting, the attack is unlikely to be mitigated with restrictive machine limits without also impacting the usefulness of the machines themselves.
* The attack can also be performed in the other direction by a malicious middle relay against clients. In this case, it would disable/degrade padding on all of the client circuits. However, we regard this variant as less impactful than client->relay attacks, in part because this attack is harder to scale.
Some initial recommendations:
* Higher severity logging when exceeding `circpad_global_max_padding_pct`.
* The consensus-provided global limits could be enforced *per circuit* instead of truly globally. The root of the problem is shared state across circuits. This would still make the global padding limits useful, since in the future there might be multiple machines active throughout the lifetime of a circuit.
* Based on the information from @pulls , the `CircuitPaddingDisabled` (see issue [28693](https://gitlab.torproject.org/legacy/trac/-/issues/28693)) switch might be sufficient as an emergency fail-safe in case of network-wide padding problems.
I would like to thank @pulls for walking me through the various aspects of related network behavior and providing other helpful suggestions.Mike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/31653Padding cells sent with 0ms delay cause circuit failures2023-06-09T13:27:09ZpullsPadding cells sent with 0ms delay cause circuit failuresBelow appears to be a bug that results in a closed circuit due to a relay sending a padding cell (RELAY_COMMAND_DROP) to the client.
At links below you find code for two circuit padding machines:
1. circpad_machine_client_close_circuit_...Below appears to be a bug that results in a closed circuit due to a relay sending a padding cell (RELAY_COMMAND_DROP) to the client.
At links below you find code for two circuit padding machines:
1. circpad_machine_client_close_circuit_minimal
2. circpad_machine_relay_close_circuit_minimal
Machine 1 runs on a client on CIRCUIT_PURPOSE_C_GENERAL circuits with 3 hops as soon as CIRCPAD_CIRC_OPENED: the only thing it does is set a timer 1000s in the future and waits for the timer to expire. The purpose of the machine is to negotiate padding with the relay to activate Machine 2 on the middle relay.
Machine 2 runs at the middle relay and repeatedly sends a padding cell to the client 1 usec after the event CIRCPAD_EVENT_NONPADDING_SENT. In other words, for every non-padding cell we directly add a padding cell. At the client, this causes `relay_decrypt_cell(): Incoming cell at client not recognized. Closing.` for all circuits triggering Machine 2 at the relay. Closing a circuit by injecting padding cells seems unintended.
Here is part of a log from a client on info level showing how the machine registers, negotiates with the relay (starting Machine 2 at the relay), eventually fails to decrypt, and the circuit closes.
```
Sep 05 10:32:19.000 [info] circpad_setup_machine_on_circ(): Registering machine client_close_circuit_minimal to origin circ 3 (5)
Sep 05 10:32:19.000 [info] circpad_node_supports_padding(): Checking padding: supported
Sep 05 10:32:19.000 [info] circpad_negotiate_padding(): Negotiating padding on circuit 3 (5), command 2
Sep 05 10:32:19.000 [info] link_apconn_to_circ(): Looks like completed circuit to [scrubbed] does allow optimistic data for connection to [scrubbed]
Sep 05 10:32:19.000 [info] connection_ap_handshake_send_begin(): Sending relay cell 0 on circ 3296464733 to begin stream 20575.
Sep 05 10:32:19.000 [info] connection_ap_handshake_send_begin(): Address/port sent, ap socket 13, n_circ_id 3296464733
Sep 05 10:32:19.000 [info] rep_hist_note_used_port(): New port prediction added. Will continue predictive circ building for 2355 more seconds.
Sep 05 10:32:19.000 [info] connection_edge_process_inbuf(): data from edge while in 'waiting for circuit' state. Leaving it on buffer.
Sep 05 10:32:19.000 [info] exit circ (length 3): $[manually-scrubbed](open) $[manually-scrubbed](open) $[manually-scrubbed](open)
Sep 05 10:32:19.000 [info] pathbias_count_use_attempt(): Used circuit 3 is already in path state use attempted. Circuit is a General-purpose client currently open.
Sep 05 10:32:19.000 [info] link_apconn_to_circ(): Looks like completed circuit to [scrubbed] does allow optimistic data for connection to [scrubbed]
Sep 05 10:32:19.000 [info] connection_ap_handshake_send_begin(): Sending relay cell 0 on circ 3296464733 to begin stream 20576.
Sep 05 10:32:19.000 [info] connection_ap_handshake_send_begin(): Address/port sent, ap socket 14, n_circ_id 3296464733
Sep 05 10:32:19.000 [info] circpad_deliver_recognized_relay_cell_events(): Got padding cell on origin circuit 3.
Sep 05 10:32:20.000 [info] relay_decrypt_cell(): Incoming cell at client not recognized. Closing.
Sep 05 10:32:20.000 [info] circuit_receive_relay_cell(): relay crypt failed. Dropping connection.
Sep 05 10:32:20.000 [info] command_process_relay_cell(): circuit_receive_relay_cell (backward) failed. Closing.
Sep 05 10:32:20.000 [info] circpad_circuit_machineinfo_free_idx(): Freeing padding info idx 0 on circuit 3 (23)
Sep 05 10:32:20.000 [info] circpad_node_supports_padding(): Checking padding: supported
Sep 05 10:32:20.000 [info] circpad_negotiate_padding(): Negotiating padding on circuit 3 (23), command 1
Sep 05 10:32:20.000 [info] pathbias_send_usable_probe(): Sending pathbias testing cell to 0.233.9.53:25 on stream 20577 for circ 3.
Sep 05 10:32:20.000 [info] relay_decrypt_cell(): Incoming cell at client not recognized. Closing.
Sep 05 10:32:20.000 [info] circuit_receive_relay_cell(): relay crypt failed. Dropping connection.
Sep 05 10:32:20.000 [info] command_process_relay_cell(): circuit_receive_relay_cell (backward) failed. Closing.
Sep 05 10:32:20.000 [info] pathbias_send_usable_probe(): Got pathbias probe request for circuit 3 with outstanding probe
Sep 05 10:32:20.000 [info] pathbias_check_close(): Circuit 3 closed without successful use for reason 2. Circuit purpose 23 currently 1,open. Len 3.
Sep 05 10:32:20.000 [info] circuit_mark_for_close_(): Circuit 3296464733 (id: 3) marked for close at src/core/or/command.c:582 (orig reason: 2, new reason: 0)
Sep 05 10:32:20.000 [info] connection_edge_destroy(): CircID 0: At an edge. Marking connection for close.
Sep 05 10:32:20.000 [info] connection_edge_destroy(): CircID 0: At an edge. Marking connection for close.
Sep 05 10:32:20.000 [info] circuit_free_(): Circuit 0 (id: 3) has been freed.
```
If we delay sending the padding from the relay I cannot trigger the bug (see commented out fix in the Machine 2 function). With the code below, the code triggers on every circuit that has the machine negotiated.
Code: https://github.com/pylls/tor/tree/circuit-padding-relay-padding-bug (https://github.com/pylls/tor/blob/circuit-padding-relay-padding-bug/src/core/or/circuitpadding_machines.c#L460, as well as circuitpadding_machines.h and registration in circpad_machines_init() of circuitpadding.c).Mike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40422[CircuitPadding] circpad_add_matching_machines() should be called when a cir...2023-06-09T13:26:45ZJaym[CircuitPadding] circpad_add_matching_machines() should be called when a circuit has opened.### Summary
The circuit padding framework supports negotiating padding upon various events. Among them, CIRCPAD_CIRC_OPENED states that a given padding machine should be applied to a circuit when a circuit has opened.
However, no code ...### Summary
The circuit padding framework supports negotiating padding upon various events. Among them, CIRCPAD_CIRC_OPENED states that a given padding machine should be applied to a circuit when a circuit has opened.
However, no code seems to trigger this mechanism. When a circuit has built, the function circpad_machine_event_circ_built() is called and checks whether some machine may be removed/added to the circuit. However, at this stage of the circuit building process, the circuit has built but is not marked as open yet.
### Bug
If some machine uses `client_machine->conditions.apply_state_mask = CIRCPAD_CIRC_OPENED;` the machine would only be applied when another event than a circ building/opening triggers the function circpad_add_matching_machines() (e.g., ap conn links a stream, or the circ purpose changes from general to something else).
### What is the expected behavior?
When circuituse.c calls circuit_has_opened(), it should also call the circpad module; e.g., a new function circpad_machine_event_circ_opened() that checks for adding machine to the circuit.
### Environment
Running a version forked from 0.4.5.7
### Relevant logs and/or screenshots
Contains some logs showing a call to circpad_machine_event_circ_built() while the circuit is still marked as building. Also contains custom logs:
```Jun 30 11:23:50.000 [info] circuit_finish_handshake(): Finished building circuit hop:
Jun 30 11:23:50.000 [info] internal (high-uptime) circ (length 3, last hop test000a): $22BA781A60C0CBB7FFAEA8858128427F67F60038(open) $7684DE04DCBB44538554E2CD1D14CDF836D5AF4D(open) $C7ADB1DBCE99F0B2ED2812B1953E4986EE9846DB(open)
Jun 30 11:23:50.000 [debug] dispatch_send_msg_unchecked(): Queued: ocirc_cevent (<gid=7 evtype=2 reason=0 onehop=0>) from or, on ocirc.
Jun 30 11:23:50.000 [debug] dispatcher_run_msg_cbs(): Delivering: ocirc_cevent (<gid=7 evtype=2 reason=0 onehop=0>) from or, on ocirc:
Jun 30 11:23:50.000 [debug] dispatcher_run_msg_cbs(): Delivering to btrack.
Jun 30 11:23:50.000 [debug] btc_cevent_rcvr(): CIRC gid=7 evtype=2 reason=0 onehop=0
Jun 30 11:23:50.000 [debug] circuit_build_times_add_time(): Adding circuit build time 43
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking circuit purpose, 5
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking condition state mask 21 vs condition: 2
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking circuit purpose, 5
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking circuit purpose, 5
Jun 30 11:23:50.000 [debug] circpad_machine_event_circ_built(): Circpad module event circ built -- circ state: 0
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking circuit purpose, 5
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking condition state mask 21 vs condition: 2
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking circuit purpose, 5
Jun 30 11:23:50.000 [debug] circpad_machine_conditions_apply(): Checking circuit purpose, 5
Jun 30 11:23:50.000 [debug] invoke_plugin_operation_or_default(): Plugin found for caller calling a plugin in the circpad module when a circuit has built
Jun 30 11:23:50.000 [info] circpad_dropmark_activate_when_built(): Looks like the client_dropmark_def machine does not exist over this circuit
Jun 30 11:23:50.000 [debug] plugin_run(): Plugin execution returned -2147483648
Jun 30 11:23:50.000 [debug] plugin_run(): vm error message: (null)
Jun 30 11:23:50.000 [info] entry_guards_note_guard_success(): Recorded success for primary confirmed guard test002r ($22BA781A60C0CBB7FFAEA8858128427F67F60038)
Jun 30 11:23:50.000 [debug] dispatch_send_msg_unchecked(): Queued: ocirc_state (<gid=7 state=4 onehop=0>) from or, on ocirc.
Jun 30 11:23:50.000 [debug] dispatcher_run_msg_cbs(): Delivering: ocirc_state (<gid=7 state=4 onehop=0>) from or, on ocirc:
Jun 30 11:23:50.000 [debug] dispatcher_run_msg_cbs(): Delivering to btrack.
Jun 30 11:23:50.000 [debug] btc_state_rcvr(): CIRC gid=7 state=4 onehop=0
Jun 30 11:23:50.000 [info] circuit_build_no_more_hops(): circuit built!
Jun 30 11:23:50.000 [info] pathbias_count_build_success(): Got success count 3.000000/3.000000 for guard test002r ($22BA781A60C0CBB7FFAEA8858128427F67F60038)
Jun 30 11:23:50.000 [debug] circuit_has_opened(): calling circuit_has_opened()
```
### Possible fixes
Add a new function circpad_machine_event_circ_opened() called from circuituse.c when the circuit has opened.Tor: 0.4.8.x-freezeMike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40196Deprecate rust-in-Tor build features for now, recommend arti rust implemntati...2021-01-08T19:52:25ZNick MathewsonDeprecate rust-in-Tor build features for now, recommend arti rust implemntation insteadWe still like Rust, but right now all the good Rust work is happening in @tpo/core/arti
The existing Rust-in-tor build system is just confusing people, and we should probably deprecate it for now. We can return to it if we want to do a...We still like Rust, but right now all the good Rust work is happening in @tpo/core/arti
The existing Rust-in-tor build system is just confusing people, and we should probably deprecate it for now. We can return to it if we want to do arti/tor integration in the future.Tor: 0.4.6.x-freezeNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40206Include the IPv6 address in the descriptor that is going to be tested for sel...2021-01-13T14:53:55Zs7rInclude the IPv6 address in the descriptor that is going to be tested for self reachabilityFor IPv4 we log the address for which we test self reachability. Very useful for debugging.
Since we have IPv6 autodiscovery now, and we try hard to better support IPv6 (yey) we should include in this log line also the IPv6 address that...For IPv4 we log the address for which we test self reachability. Very useful for debugging.
Since we have IPv6 autodiscovery now, and we try hard to better support IPv6 (yey) we should include in this log line also the IPv6 address that is going to be tested for self reachability (if one exists of course and the relay/bridge is not IPv4 only).
Relays / bridges are not marked as running by directory authorities if they advertise an IPv6 address but are not reachable on it, so it makes full sense for the operators to know which address is being tested.Tor: 0.4.5.x-stableDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40265Rebuild fallbackdir list January 20212021-04-15T12:51:30ZGeorge KadianakisRebuild fallbackdir list January 2021Tor: 0.4.6.x-stableGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/tpo/core/tor/-/issues/402860.4.5 spends 10 minutes at 100% cpu when loading old cached-descriptors* files2021-03-19T17:20:54ZRoger Dingledine0.4.5 spends 10 minutes at 100% cpu when loading old cached-descriptors* filesI started up an old relay that had cached-descriptors* files from a couple years ago.
It maxed out its cpu and sat there for around 10 minutes on a fast cpu, doing whatever it was doing. It ignored kill attempts other than kill -9. Even...I started up an old relay that had cached-descriptors* files from a couple years ago.
It maxed out its cpu and sat there for around 10 minutes on a fast cpu, doing whatever it was doing. It ignored kill attempts other than kill -9. Eventually it finished and things were fine after that.
I tested and both Tor clients and relays encounter the issue. Tor 0.4.4 does not encounter it. Maybe it is related to the tor#40281 / tor#40221 changes?
It's not the end of the world, since it only needs to happen once per upgrade, and maybe if you have recent enough files it doesn't happen to you. But I would feel pretty sorry for people who don't have great CPUs and who get bit by this one.
I'll put the cached-* files up somewhere once I have a ticket number.Tor: 0.4.5.x-post-stableNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40297Core Tor 0.4.5.6 claims "OpenSSL has been built without thread support" on st...2021-04-13T21:06:21ZpkreuztCore Tor 0.4.5.6 claims "OpenSSL has been built without thread support" on static Mingw buildBuilding on Linux for static win32 and win64. This didn't happen on previous versions, which built correctly with same OpenSSL. Also, this error arises on make step, configure doesn't report any problem:
```
src/lib/crypt_ops/crypto_ope...Building on Linux for static win32 and win64. This didn't happen on previous versions, which built correctly with same OpenSSL. Also, this error arises on make step, configure doesn't report any problem:
```
src/lib/crypt_ops/crypto_openssl_mgt.c:135:2: error: #error "OpenSSL has been built without thread support. Tor requires an OpenSSL library with thread support enabled."
[...]
make[1]:*** [Makefile:16657: src/lib/crypt_ops/libtor_crypt_ops_a-crypto_openssl_mgt.o] Error 1
```Tor: 0.4.5.x-post-stableNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40352Nonauthoritative directory does not accept posted server descriptors2021-10-19T12:31:25ZTommaso GragnatoNonauthoritative directory does not accept posted server descriptors```[WARN] http status 400 ("Nonauthoritative directory does not accept posted server descriptors") response from dirserver 185.107.47.215:9001. Malformed rendezvous descriptor? [6 duplicates hidden]```
I suspect this is because [185.107...```[WARN] http status 400 ("Nonauthoritative directory does not accept posted server descriptors") response from dirserver 185.107.47.215:9001. Malformed rendezvous descriptor? [6 duplicates hidden]```
I suspect this is because [185.107.47.215](https://metrics.torproject.org/rs.html#details/45E9240AD4ECE01793A1977C1260503B2C2C861F) is running 0.4.6 alpha and does not understand descriptors for v2 onions. Hence the 400.
https://gitlab.torproject.org/tpo/core/tor/-/issues/40266#note_2724898
Should this be handled gracefully? What's the phase out plan?Tor: 0.4.7.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40342Tor 0.4.5.7 minor log spam; Unable to find IPv6 address for ORPort2021-03-23T18:53:26ZerTor 0.4.5.7 minor log spam; Unable to find IPv6 address for ORPortThats after 6 hours uptime. Initial remark in logs would be ok. Repeating it over and over again after that, this kinda isnt. So there is no ipv6 address configured. Is that illegal? Can you still function? Take a hint, will you? Just m...Thats after 6 hours uptime. Initial remark in logs would be ok. Repeating it over and over again after that, this kinda isnt. So there is no ipv6 address configured. Is that illegal? Can you still function? Take a hint, will you? Just my 2 cents.
```
Mar 17 15:19:36.825 [notice] Tor 0.4.5.7 running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1i, Zlib 1.2.11, Liblzma 5.2.5, Libzstd 1.4.8 and Glibc 2.31 as libc.
...
Mar 17 15:20:01.000 [notice] Unable to find IPv4 address for ORPort 9001. You might want to specify IPv6Only to it or set an explicit address or set Address.
...
Mar 17 16:20:02.000 [notice] Unable to find IPv6 address for ORPort 9001. You might want to specify IPv4Only to it or set an explicit address or set Address. [62 similar message(s) suppressed in last 3660 seconds]
Mar 17 17:20:03.000 [notice] Unable to find IPv6 address for ORPort 9001. You might want to specify IPv4Only to it or set an explicit address or set Address. [59 similar message(s) suppressed in last 3540 seconds]
Mar 17 18:20:03.000 [notice] Unable to find IPv6 address for ORPort 9001. You might want to specify IPv4Only to it or set an explicit address or set Address. [59 similar message(s) suppressed in last 3600 seconds]
Mar 17 19:20:03.000 [notice] Unable to find IPv6 address for ORPort 9001. You might want to specify IPv4Only to it or set an explicit address or set Address. [59 similar message(s) suppressed in last 3540 seconds]
Mar 17 20:20:03.000 [notice] Unable to find IPv6 address for ORPort 9001. You might want to specify IPv4Only to it or set an explicit address or set Address. [59 similar message(s) suppressed in last 3540 seconds]
Mar 17 21:20:03.000 [notice] Unable to find IPv6 address for ORPort 9001. You might want to specify IPv4Only to it or set an explicit address or set Address. [59 similar message(s) suppressed in last 3540 seconds]
```Tor: 0.4.5.x-post-stableDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40303rate limit a warn message: "[warn] Error binding network socket: Address alre...2021-04-16T14:12:36Ztoralfrate limit a warn message: "[warn] Error binding network socket: Address already in use"It tends to flood the warn.log under certain circumstances.It tends to flood the warn.log under certain circumstances.Tor: 0.4.6.x-stableDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40317Cannot SAVECONF when the seccomp sandbox is enabled2022-07-07T00:47:10ZanonymCannot SAVECONF when the seccomp sandbox is enabled### Summary
Issuing a `SAVECONF` to `tor` when the seccomp sandbox is enabled fails.
### Steps to reproduce:
1. Start `tor` with `Sandbox 1` set and controller access
2. Send `SAVECONF` via the controller
### What is the current bug ...### Summary
Issuing a `SAVECONF` to `tor` when the seccomp sandbox is enabled fails.
### Steps to reproduce:
1. Start `tor` with `Sandbox 1` set and controller access
2. Send `SAVECONF` via the controller
### What is the current bug behavior?
Sending a `SAVECONF` currently results in this answer:
```
250 OK
551 Unable to write configuration to disk.
250 closing connection
```
and `tor` logs:
```
Mar 05 11:31:28.000 [warn] sandbox_intern_string(): Bug: No interned sandbox parameter found for /etc/tor/torrc.orig.1 (on Tor 0.4.5.6 )
Mar 05 11:31:28.000 [notice] Renaming old configuration file to "/etc/tor/torrc.orig.1"
Mar 05 11:31:28.000 [warn] sandbox_intern_string(): Bug: No interned sandbox parameter found for /etc/tor/torrc.orig.1 (on Tor 0.4.5.6 )
Mar 05 11:31:28.000 [warn] Couldn't rename configuration file "/etc/tor/torrc" to "/etc/tor/torrc.orig.1": Operation not permitted
```
and unsurprisingly the current `tor` configuration was not saved to `torrc`.
### What is the expected behavior?
`SAVECONF` should work even when the seccomp sandbox is enabled.
### Environment
I tried with both `tor` 0.4.4.6 and 0.4.5.6 on an Debian Buster, using your `.deb`:s.
### Relevant logs and/or screenshots
Except the log I posted above, sometimes I saw this instead but I don't know why it's different:
```
Mar 05 12:06:00.000 [warn] sandbox_intern_string(): Bug: No interned sandbox parameter found for /etc/tor/torrc.tmp (on Tor 0.4.4.6 )
Mar 05 12:06:00.000 [warn] Couldn't open "/etc/tor/torrc.tmp" (/etc/tor/torrc) for writing: Operation not permitted
```Tor: 0.4.5.x-post-stableNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40384move doxygen docs to GitLab CI or deprecate2021-08-30T17:59:27Zanarcatmove doxygen docs to GitLab CI or deprecatein tpo/tpa/team#40225, i was asked to "remove or disable Tor jenkins builders". while doing that, I noticed that doxgen docs are not currently build in a known location in GitLab CI:
https://src-ref.docs.torproject.org/tor/index.html
s...in tpo/tpa/team#40225, i was asked to "remove or disable Tor jenkins builders". while doing that, I noticed that doxgen docs are not currently build in a known location in GitLab CI:
https://src-ref.docs.torproject.org/tor/index.html
so retiring those would mean leaving that stale website in place for the foreseeable future.
could you move that to GitLab CI and GitLab pages? the following should get you started:
https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/gitlab/#publishing-gitlab-pages
if not, do let me know what's your blocker. i *believe* the resulting URL for core tor would be:
https://tpo.pages.torproject.net/core/tor/
once that is done we can retire the jenkins job, and then provide a redirect to the gitlab pages.
makes sense?Retire JenkinsAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40418Man page no longer accurate for onion authorization instructions2022-05-09T16:35:19ZnyxnorMan page no longer accurate for onion authorization instructions[2019 TPO docs](https://2019.www.torproject.org/docs/tor-manual.html.en)
Outdated:
> Client Authorization
> Revoking a client can be done by removing their ".auth" file, however the revocation will be in effect only after the tor proce...[2019 TPO docs](https://2019.www.torproject.org/docs/tor-manual.html.en)
Outdated:
> Client Authorization
> Revoking a client can be done by removing their ".auth" file, however the revocation will be in effect only after the tor process gets restarted even if a SIGHUP takes place.
Revoking a key does work with sighup now.Developer portalSilvio RhattoSilvio Rhattohttps://gitlab.torproject.org/tpo/core/tor/-/issues/40427enable TCP timestamps on outgoing ORcon2021-11-01T17:10:27ZpseudonymisaTorenable TCP timestamps on outgoing ORcon### Summary
Make use of TCP timestamps, enable it via socks options on outgoing connections. [Defined in RFC 1323](https://datatracker.ietf.org/doc/html/rfc1323#section-4).
Tor could benefit from some better TCP congestion flow control...### Summary
Make use of TCP timestamps, enable it via socks options on outgoing connections. [Defined in RFC 1323](https://datatracker.ietf.org/doc/html/rfc1323#section-4).
Tor could benefit from some better TCP congestion flow control? The Client or relay does not know what TCP congestion algorithm the other peer may use.
> TCP timestamps are enabled by default In Linux kernel.
It was once disabled in some distros, because the bad implementation of timestamp start time was your system uptime and this fingerprinting could leak your uptime on every connection. This was fixed 2 decades ago. So why not use it now for all and not only some? All Tor Connection should look most identical for fingerprinting reasons.
Why not use it? Worst thing to happen is, that it could add 8 extra bytes of TCP header in total.
### What is the expected behavior?
TCP connections should be always enabling timestamps, to make fingerprinting harder.
WiP: [+TCP_TIMESTAMPS.patch](/uploads/fcbb3a72f3cbfdb91eab7b8e26af26f3/+TCP_TIMESTAMPS.patch)Mike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40430Modify default torrc text for PublishServerDescriptor2021-07-08T18:19:04ZCecylia BocovichModify default torrc text for PublishServerDescriptorIn the default torrc file, we see:
```
## By default, Tor will advertise your bridge to users through various
## mechanisms like https://bridges.torproject.org/. If you want to run
## a private bridge, for example because you'll give out...In the default torrc file, we see:
```
## By default, Tor will advertise your bridge to users through various
## mechanisms like https://bridges.torproject.org/. If you want to run
## a private bridge, for example because you'll give out your bridge
## address manually to your friends, uncomment this line:
#PublishServerDescriptor 0
```
However, we've noticed that some default Tor and Orbot bridges are not publishing their descriptors and therefore we aren't getting any metrics from them. This is exactly the use-case that `BridgeDistribution none` was meant to handle. In general, we recommend that people running private (or default) bridges that aren't meant for BridgeDB to configure their torrc file to have:
```
PublishServerDescriptor 1
BridgeDistribution none
```
See also the text in the torrc manpage for `PublishServerDescriptor`:
```
PublishServerDescriptor 0|1|v3|bridge,...
This option specifies which descriptors Tor will publish when acting as a relay. You
can choose multiple arguments, separated by commas.
If this option is set to 0, Tor will not publish its descriptors to any directories.
(This is useful if you’re testing out your server, or if you’re using a Tor controller
that handles directory publishing for you.) Otherwise, Tor will publish its descriptors
of all type(s) specified. The default is "1", which means "if running as a relay or
bridge, publish descriptors to the appropriate authorities". Other possibilities are
"v3", meaning "publish as if you’re a relay", and "bridge", meaning "publish as if
you’re a bridge".
```
I think we should change the default torrc text to match the manpage and say something to the effect of "if you want to, you can avoid publishing your descriptors, but we recommend that you do and that you set `BridgeDistribution none` if you don't want your bridge distributed over BridgeDB. That way we can collect metrics."Tor: 0.4.7.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40445tor_tls_finish_handshake warning spamming logs during relay operation2022-07-07T00:47:11Zharpiator_tls_finish_handshake warning spamming logs during relay operation### Summary
The following message is rapidly filling my notice-level logs during the Tor middle relay operation:
```
[warn] tor_tls_finish_handshake: Bug: For some reason, wasV2Handshake didn't get set. Fixing that. (on Tor 0.4.5.9 )
``...### Summary
The following message is rapidly filling my notice-level logs during the Tor middle relay operation:
```
[warn] tor_tls_finish_handshake: Bug: For some reason, wasV2Handshake didn't get set. Fixing that. (on Tor 0.4.5.9 )
```
The log file is growing fast. I've sent a bug report to OpenBSD via email, but I'm not sure they received it - no response so far. That's why I brought it here. Maybe it's reproducible upstream too.
### Steps to reproduce:
1. Install the tor package with "pkg_add tor"
2. Edit /etc/tor/torrc as in this gist: https://gist.github.com/o-alquimista/ba5b1043cb1be05eb76e0dd7831bf44d
3. Start tor service
The tor service may fail to start due to "permission denied" when writing the notice log file. I had to manually create the /var/log/tor directory and set its ownership to the "_tor" user.
4. Watch the notices.log file. In spite of the warnings, the relay runs without any noticeable problems. You can see it here: https://metrics.torproject.org/rs.html#details/3FA93D41E9A7C4C47B77C0D7F617999B6D5D0B62
### Environment
- Tor 0.4.5.9
- OpenBSD 6.9 arm64 (-stable)
- The device is a Raspberry Pi 4 Model B Rev 1.2
- Installation method is pkg_add (OpenBSD package)Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40496We log when we stop having enough dir info, but we never log when we resume h...2021-11-03T22:53:05ZRoger DingledineWe log when we stop having enough dir info, but we never log when we resume having enoughWhen something goes wrong with our ability to use the network, we get a notice-level log like this one:
```
Oct 23 05:21:03.818 [notice] Our directory information is no longer up-to-date enough to build circuits: We're missing descriptor...When something goes wrong with our ability to use the network, we get a notice-level log like this one:
```
Oct 23 05:21:03.818 [notice] Our directory information is no longer up-to-date enough to build circuits: We're missing descriptors for 1/2 of our primary entry guards (total microdescriptors: 6432/6432). That's ok. We will try to fetch missing descriptors soon.
```
But when things recover, that message is only at info-level:
```
Oct 23 05:21:04.822 [info] We now have enough directory information to build circuits.
```
The original plan is described in the comments in update_router_have_minimum_dir_info():
```
/* a) make us log when we next complete a circuit, so we know when Tor
* is back up and usable, and b) disable some activities that Tor
* should only do while circuits are working, like reachability tests
* and fetching bridge descriptors only over circuits. */
note_that_we_maybe_cant_complete_circuits();
```
but then we lost that notice-level log in commit eee62e13 (in Tor 0.3.5.1-alpha):
```
- log_notice(LD_GENERAL,
- "Tor has successfully opened a circuit. "
- "Looks like client functionality is working.");
[...]
+ log_info(LD_GENERAL,
+ "Tor has successfully opened a circuit. "
+ "Looks like client functionality is working.");
```
as part of the shift in #14950 to reduce log clutter.
We should make sure there is a corresponding "we're back" message for each "we can't work yet" message that we write.https://gitlab.torproject.org/tpo/core/tor/-/issues/25630Bug: 3-hop circuit 0x55e55d447270 with purpose 5 has no guard state2021-11-06T14:54:53ZmeejahBug: 3-hop circuit 0x55e55d447270 with purpose 5 has no guard stateAfter discussion on #tor-dev, arma told me to file this; I am hitting this for every circuit my exit-scanner creates when using tor master (0eed0899cdeadd84).
The circuits created all start with a current guard (but probably not any of ...After discussion on #tor-dev, arma told me to file this; I am hitting this for every circuit my exit-scanner creates when using tor master (0eed0899cdeadd84).
The circuits created all start with a current guard (but probably not any of "my" current guards). I've tried with both purpose=general and purpose=controller but the result is the same (except for the 5 vs 21 in the error message)Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40629Allow ignoring of SIGINT2022-06-23T21:13:51ZtlaAllow ignoring of SIGINT### Summary
Add an option (.e.g `--IgnoreSigint 1`) which allows to ignore `SIGINT`.
iOS has a feature which enables apps to keep running in the background for a certain amount of time:
https://developer.apple.com/documentation/uikit/...### Summary
Add an option (.e.g `--IgnoreSigint 1`) which allows to ignore `SIGINT`.
iOS has a feature which enables apps to keep running in the background for a certain amount of time:
https://developer.apple.com/documentation/uikit/uiapplication/1623031-beginbackgroundtask
However, even when we're making use of that, iOS is sending `SIGINT` to the app process, as soon as the user swipes away the app. (Sends it into background.)
Tor is currently hardcoded to stop working, when it receives that `SIGINT`:
https://gitlab.torproject.org/tpo/core/tor/-/blob/main/src/app/main/main.c#L223-228
### What is the expected behavior?
When the mentioned configuration option is set, Tor just ignores the `SIGINT` and continues running, to enable processing in the background.Tor: 0.4.8.x-freezeAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40706Tor gets killed by OOM killer2022-11-01T13:26:30ZanongTor gets killed by OOM killerHi,
my relay has been killed a few times by my OS's OOM killer in the last few days.
https://metrics.torproject.org/rs.html#details/F2D6EB211744D41DC41CCB62BF1C3246D24B42A2
`Tor 0.4.7.10 running on Linux with Libevent 2.1.12-stable, O...Hi,
my relay has been killed a few times by my OS's OOM killer in the last few days.
https://metrics.torproject.org/rs.html#details/F2D6EB211744D41DC41CCB62BF1C3246D24B42A2
`Tor 0.4.7.10 running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1n, Zlib 1.2.11, Liblzma 5.2.5, Libzstd 1.4.8 and Glibc 2.31 as libc.`
MaxMemInQueues was set to 3072 MB when the last crash happened. System is an Debian LXC container with currently 10 gigabytes of RAM.
OS is `Debian GNU/Linux 11 (bullseye)`
Tor is installed via official Debian repositories.
```
Oct 27 22:47:27 Tor systemd[1]: tor@default.service: A process of this unit has been killed by the OOM killer.
Oct 27 22:47:27 Tor systemd[1]: tor@default.service: Main process exited, code=killed, status=9/KILL
Oct 27 22:47:27 Tor systemd[1]: tor@default.service: Failed with result 'oom-kill'.
Oct 27 22:47:27 Tor systemd[1]: tor@default.service: Consumed 2d 13h 43min 1.409s CPU time.
Oct 27 22:47:28 Tor systemd[1]: tor@default.service: Scheduled restart job, restart counter is at 4.
Oct 27 22:47:28 Tor systemd[1]: Stopped Anonymizing overlay network for TCP.
Oct 27 22:47:28 Tor systemd[1]: tor@default.service: Consumed 2d 13h 43min 1.409s CPU time.
Oct 27 22:47:28 Tor systemd[1]: Starting Anonymizing overlay network for TCP...
```
MaxMemInQueues is now not specifically defined anymore, so automatically determined.
Complete startup sequence from current run:
```
Oct 27 22:47:28 Tor tor[6405]: Oct 27 22:47:28.248 [notice] Tor 0.4.7.10 running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1n, Zlib 1.2.11, Liblzma 5.2.5, Libzstd 1.4.8 and Glibc 2.31 as libc.
Oct 27 22:47:28 Tor tor[6405]: Oct 27 22:47:28.248 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://support.torproject.org/faq/staying-anonymous/
Oct 27 22:47:28 Tor tor[6405]: Oct 27 22:47:28.248 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Oct 27 22:47:28 Tor tor[6405]: Oct 27 22:47:28.248 [notice] Read configuration file "/etc/tor/torrc".
Oct 27 22:47:28 Tor tor[6405]: Oct 27 22:47:28.249 [warn] Skipping obsolete configuration option "SocksListenAddress".
Oct 27 22:47:28 Tor tor[6405]: Oct 27 22:47:28.251 [notice] Based on detected system memory, MaxMemInQueues is set to 4096 MB. You can override this by setting MaxMemInQueues by hand.
Oct 27 22:47:28 Tor tor[6405]: Configuration was valid
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.265 [notice] Tor 0.4.7.10 running on Linux with Libevent 2.1.12-stable, OpenSSL 1.1.1n, Zlib 1.2.11, Liblzma 5.2.5, Libzstd 1.4.8 and Glibc 2.31 as libc.
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.265 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://support.torproject.org/faq/staying-anonymous/
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.265 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.265 [notice] Read configuration file "/etc/tor/torrc".
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.265 [warn] Skipping obsolete configuration option "SocksListenAddress".
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.266 [notice] Based on detected system memory, MaxMemInQueues is set to 4096 MB. You can override this by setting MaxMemInQueues by hand.
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] I think we have 8 CPUS, but only 4 of them are available. Telling Tor to only use 4. You can override this with the NumCPUs option
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] You configured a non-loopback address '192.168.100.9:9050' for SocksPort. This allows everybody on your local network to use your machine as a proxy. Make sure this is wha>
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opening Socks listener on 192.168.100.9:9050
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opened Socks listener connection (ready) on 192.168.100.9:9050
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opening Control listener on 127.0.0.1:9051
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opened Control listener connection (ready) on 127.0.0.1:9051
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opening OR listener on 0.0.0.0:443
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opened OR listener connection (ready) on 0.0.0.0:443
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opening OR listener on [2003:a:1504:6000:a03a:d1ff:fea1:8062]:443
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opened OR listener connection (ready) on [2003:a:1504:6000:a03a:d1ff:fea1:8062]:443
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opening Directory listener on 0.0.0.0:80
Oct 27 22:47:28 Tor tor[6407]: Oct 27 22:47:28.267 [notice] Opened Directory listener connection (ready) on 0.0.0.0:80
```
Which more information is needed? Can you help me find the cause of this issue?https://gitlab.torproject.org/tpo/core/tor/-/issues/40676ExitPolicy should apply to already established outbound connections (with a c...2023-08-25T16:55:41ZcypherpunksExitPolicy should apply to already established outbound connections (with a config option, off by default)To reduce the impact of tor running out of free TCP source ports (see pending comment in #26646) we added a reject entry for the destination causing most outbound TCP connections to the ExitPolicy and restarted tor.
```
ExitPolicy reje...To reduce the impact of tor running out of free TCP source ports (see pending comment in #26646) we added a reject entry for the destination causing most outbound TCP connections to the ExitPolicy and restarted tor.
```
ExitPolicy reject 1.2.3.4:* <<<< added to avoid outbound connections to this target
ExitPolicy accept *:80
ExitPolicy accept *:443
ExitPolicy reject *:*
```
Expected: Tor should not create any connections to 1.2.3.4
Even after changing the torrc and restarting tor we see established TCP connections to 1.2.3.4,
this is unexpected.Tor: 0.4.8.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40711Tor with new ipv6 only works after a restart2022-12-16T18:26:07ZGusTor with new ipv6 only works after a restartA relay operator reported that every day their relay changes the IPv4 and IPv6 addresses. But while Tor with IPv4 works automatically, for IPv6 connection, they need to restart the service.
```
Auto-discovered IPv6 address [1111:::ffff]...A relay operator reported that every day their relay changes the IPv4 and IPv6 addresses. But while Tor with IPv4 works automatically, for IPv6 connection, they need to restart the service.
```
Auto-discovered IPv6 address [1111:::ffff]:9001 has not been found reachable. However, IPv4 address is reachable. Publishing server descriptor without IPv6 address. [2 similar message(s) suppressed in last 2400 seconds])
```
#### Tor version
0.4.7.10.
#### torrc
```
SocksPort 0
Log notice file /var/log/tor/notices.log
DataDirectory /var/lib/tor
ControlPort xxxxx
HashedControlPassword xxxxx
ORPort 9001
Nickname justme
RelayBandwidthRate 250 KB # Throttle traffic to 100KB/s (800Kbps)
RelayBandwidthBurst 750 KB # But allow bursts up to 200KB/s (1600Kbps)
ContactInfo me@there.net
DirPort 9005
ExitRelay 0
ExitPolicy reject *:* # no exits allowed
```
#### Forum link
https://forum.torproject.net/t/ipv6-with-dynamic-prefix-behind-nat/5296https://gitlab.torproject.org/tpo/core/tor/-/issues/40715MetricsPort: inbound ORPort connections: relays vs. non-relay connections2023-09-22T23:50:13ZcypherpunksMetricsPort: inbound ORPort connections: relays vs. non-relay connectionsthis got previously submitted on 2022-10-24 https://gitlab.torproject.org/tpo/core/tor/-/issues/40194#note_2849481
but that issue got closed and asked for new specific tickets for each new metric:
From last week's relay meetup we know t...this got previously submitted on 2022-10-24 https://gitlab.torproject.org/tpo/core/tor/-/issues/40194#note_2849481
but that issue got closed and asked for new specific tickets for each new metric:
From last week's relay meetup we know that tor knows whether an incoming OR connection is from a client or from a relay without looking at the source IP address.
https://pad.riseup.net/p/tor-relay-op-meetup-o22-keep
From the metrics added in !625 (merged) we know, that the increased CPU load correlates with an increase in the rate of new inbound OR connections. This rate increases when CPU load increases on exits:
```
rate(tor_relay_connections{type="OR",state="created",direction="received"}[$__rate_interval])
```
Could you please add a label for OR connections coming from clients vs. OR connections coming from other relays?
This would allow us to confirm that exits get more new inbound connections from clients when CPU load increases.
that new label could be `src`:
```
tor_relay_connections_total{type="OR",state="created",direction="received",src="relay"}
tor_relay_connections_total{type="OR",state="created",direction="received",src="non-relay"}
tor_relay_connections{type="OR",state="opened",direction="received",src="relay"}
tor_relay_connections{type="OR",state="opened",direction="received",src="non-relay"}
```https://gitlab.torproject.org/tpo/core/tor/-/issues/40716Impelement conflux for onion services2022-11-28T14:01:05ZMike PerryImpelement conflux for onion servicesConflux is traffic splitting, and will result in increased throughput and reduced latency for onion services after a connection has been established, by routing traffic over multiple paths, or via the lowest latency path to a service.
T...Conflux is traffic splitting, and will result in increased throughput and reduced latency for onion services after a connection has been established, by routing traffic over multiple paths, or via the lowest latency path to a service.
This ticket is for the onion service pieces of conflux (https://gitlab.torproject.org/tpo/core/tor/-/issues/40593).
We will not be implementing the onion services pieces of conflux as part of that ticket. It can be done later, if any onion service sponsors care about latency or throughput.
The pieces for onion services are:
- **Negotiation**
- [ ] Protover Advertisement for Onions (24h)
- [ ] Rend circuit linking (40h)
This is specified in https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/329-traffic-splitting.txt, but we probably want to allow onion services to configure their scheduler by manually choosing either BLEST, or LowRTT, since different kinds of onion services may want to optimize for either throughput or latency.
There may be some additional work wrt making sure linked edge conns work properly, if they are handled differently for the onion service case.
Also, some shadow validation and performance testing will be needed. Maybe 40h or so of dev time (though much longer wall-clock time).https://gitlab.torproject.org/tpo/core/tor/-/issues/40717Additional metricsport stats for various stages of onionservice handshake2023-12-07T14:41:35ZMike PerryAdditional metricsport stats for various stages of onionservice handshakeIf we export additional onion service metrics such as time measurements on the HSDIR, INTRO, and REND stages of circuit setup for both client and service side, and the number of timeouts/failures there, it would help to uncover the root ...If we export additional onion service metrics such as time measurements on the HSDIR, INTRO, and REND stages of circuit setup for both client and service side, and the number of timeouts/failures there, it would help to uncover the root cause of issues like https://gitlab.torproject.org/tpo/core/tor/-/issues/40570 and related reliability and connectivity issues with onion services.
We can also export congestion control info from https://gitlab.torproject.org/tpo/core/tor/-/issues/40708 to the onionservice metrics set, too, which can help us with tuning congestion control for onion services.
We can then hook up the onionperf onion service instances to our grafana dashboard, and gather more detailed stats that way, as a supplement to the metrics that get graphed on the metrics website.https://gitlab.torproject.org/tpo/core/tor/-/issues/40718Allow setting DirPortFrontPage with DirCache 02023-09-05T12:53:08ZcypherpunksAllow setting DirPortFrontPage with DirCache 0relay meetup follow up:
Currently it is not possible to set `DirCache 0` on a relay that has set `DirPort`.
```
DirPort configured but DirCache disabled. DirPort requires DirCache.
```relay meetup follow up:
Currently it is not possible to set `DirCache 0` on a relay that has set `DirPort`.
```
DirPort configured but DirCache disabled. DirPort requires DirCache.
```https://gitlab.torproject.org/tpo/core/tor/-/issues/40735[WARN] Tried connecting to router ... identity keys were not as expected2023-11-14T16:59:05Zcypherpunks[WARN] Tried connecting to router ... identity keys were not as expectedBackground: Tor Browser 12.0, Tor 4.7.12, Windows 7, vanilla bridges.
Repeatedly getting the following log line.
```
[WARN] Tried connecting to router at *address* ID=<none> RSA_ID=*FP1*, but RSA + ed25519 identity keys were not as exp...Background: Tor Browser 12.0, Tor 4.7.12, Windows 7, vanilla bridges.
Repeatedly getting the following log line.
```
[WARN] Tried connecting to router at *address* ID=<none> RSA_ID=*FP1*, but RSA + ed25519 identity keys were not as expected: wanted *FP1* + no ed25519 key but got *FP2* + *edFP*.
```
Ideas of what happened:
* MITM
* Bridge operator reinstalled it in-between me getting the bridge and now.
What is wrong:
* Bridge should be marked as unreachable: either it is not used already and connections are doomed to spend resources for nothing, or it should not be used as something is clearly wrong with it
* There should be a way to distinguish first idea from second - my best guess is building a tunneled directory connection to bridge authority and asking "Is there a bridge *FP2* and does it listen on *address*?"https://gitlab.torproject.org/tpo/core/tor/-/issues/40739[warn] Possible compression bomb; abandoning stream.2023-11-18T19:42:40Zcomputer_freak[warn] Possible compression bomb; abandoning stream.Relay with `Tor 0.4.7.13`:
```
[warn] Possible compression bomb; abandoning stream.
[warn] Unable to decompress HTTP body (tried Zstandard compressed, on Directory connection (client reading) with 86.59.21.38:80).
```
obfs4 Bridge with ...Relay with `Tor 0.4.7.13`:
```
[warn] Possible compression bomb; abandoning stream.
[warn] Unable to decompress HTTP body (tried Zstandard compressed, on Directory connection (client reading) with 86.59.21.38:80).
```
obfs4 Bridge with `Tor 0.4.7.13`:
```
[warn] Possible compression bomb; abandoning stream.
[warn] Error while uncompressing data: bad input?
[warn] Unable to decompress HTTP body (tried Zstandard compressed, on Directory connection (client reading) with 95.214.53.221:443).
```
Relay with `Tor 0.4.8.0-alpha-dev`:
```
[warn] Possible compression bomb; abandoning stream.
[warn] Unable to decompress HTTP body (tried Zstandard compressed, on Directory connection (client reading) with 199.58.81.140:80).
```
There are no more logs on `notice` log level.
A user at the [forum](https://forum.torproject.net/t/compression-bomb-in-tor-logs/6226) has the same problem.Alexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40746Conflicting logic about whether bridges need descriptors for fetching dir inf...2023-04-12T14:42:46ZRoger DingledineConflicting logic about whether bridges need descriptors for fetching dir info from themIf you start your Tor with a pile of configured bridges but nothing cached, your Tor will sample the configured bridges to pick its ordered list of primary entry guards, and launch descriptor fetches to each of them.
But if the descript...If you start your Tor with a pile of configured bridges but nothing cached, your Tor will sample the configured bridges to pick its ordered list of primary entry guards, and launch descriptor fetches to each of them.
But if the descriptor hasn't arrived yet, while trying to bootstrap dir info you get these confusing messages in your logs:
```
Jan 31 18:56:44.928 [notice] Ignoring directory request, since no bridge nodes are available yet.
```
Things do bootstrap eventually, but it takes longer than it should, and the pile of scary log messages is scary.
What's going on here?
The way the log message comes about is that directory_get_from_dirserver() calls
```
const node_t *node = guards_choose_dirguard(dir_purpose, &guard_state);
if (node && node->ri) {
[...]
} else {
[...]
log_notice(LD_DIR, "Ignoring directory request, since no bridge "
"nodes are available yet.");
}
```
i.e. guards_choose_dirguard had better return a bridge for which we have the descriptor, or we're going to log a complaint and abort the directory fetch attempt.
But in select_primary_guard_for_circuit(), we do
```
const int need_descriptor = (usage == GUARD_USAGE_TRAFFIC);
[...]
SMARTLIST_FOREACH_BEGIN(gs->primary_entry_guards, entry_guard_t *, guard) {
[...]
if (guard->is_reachable != GUARD_REACHABLE_NO) {
if (need_descriptor && !guard_has_descriptor(guard)) {
log_info(LD_GUARD, "Guard %s does not have a descriptor",
entry_guard_describe(guard));
continue;
}
```
That is, in select_primary_guard_for_circuit() we require that the bridge have a descriptor only for the GUARD_USAGE_TRAFFIC case, but then in directory_get_from_dirserver() we expect that the bridge will always have a descriptor, even in the GUARD_USAGE_DIRGUARD case.
In normal operation this bug isn't a big deal, because it is a race to finish fetching the descriptor before we happen to pick it for asking directory info. But with the #40578 fix, where we defer fetching the descriptor if we won't use the bridge for the GUARD_USAGE_TRAFFIC case, the bug becomes more obvious.
I believe the fix is simply to always need_descriptor in select_primary_guard_for_circuit() -- meaning when we're going to launch a directory fetch we always choose among our primary guards who have descriptors already.https://gitlab.torproject.org/tpo/core/tor/-/issues/40774libtor.a: pubsub_install tor_raw_abort2024-03-20T17:17:22Zsbslibtor.a: pubsub_install tor_raw_abort### Summary
We see OONI Probe Android crashes where `pubsub_install` calls `tor_raw_abort` for tor 0.4.7.13 using libtor.a embedded into a dynamic library loaded by an Android app. As of 2023-02-09 (around when we started investigating)...### Summary
We see OONI Probe Android crashes where `pubsub_install` calls `tor_raw_abort` for tor 0.4.7.13 using libtor.a embedded into a dynamic library loaded by an Android app. As of 2023-02-09 (around when we started investigating), this issue occurred 526 times in the last 28 days and was one of the main sources of crashes for the OONI Probe Android app.
A typical stack trace obtained from the Google Play console looks like this:
```
backtrace:
#00 pc 0x0000000000089b0c .../lib64/bionic/libc.so (abort+164)
#01 pc 0x00000000013778a4 .../split_config.arm64_v8a.apk (tor_raw_abort_+12)
#02 pc 0x0000000001382150 .../split_config.arm64_v8a.apk (tor_abort_+12)
#03 pc 0x00000000012470a0 .../split_config.arm64_v8a.apk (pubsub_install+120)
#04 pc 0x0000000001247170 .../split_config.arm64_v8a.apk (tor_run_main+136)
```
We investigated this issue and manage to reproduce it initially on OONI Probe Android, then in Linux using our Go code for managing libtor.a, and finally with a pure C test case working under Linux. During this investigating we have never seen the first bootstrap failing. Rather, in some cases it took > 30 repeated bootstraps to observe the abort; in other cases, it occurred within the first 3-10 bootstraps.
I searched in the issue tracker for "pubsub", "pubsub_install", "SIGABRT", and "abort". AFAICT, there is no other open issue discussing this problem, however, I think https://gitlab.torproject.org/tpo/core/tor/-/issues/32729 may be related and ~similar.
### Steps to reproduce:
The following steps allowed me to reproduce the problem on Ubuntu 22.04.2:
1. `git clone https://gitlab.torproject.org/tpo/core/tor`
2. `cd tor`
3. `git checkout tor-0.4.7.13`
4. `git apply 004.diff` where `004.diff` is
```diff
diff --git a/src/lib/pubsub/pubsub_check.c b/src/lib/pubsub/pubsub_check.c
index 99e604d715..a5cc4b7658 100644
--- a/src/lib/pubsub/pubsub_check.c
+++ b/src/lib/pubsub/pubsub_check.c
@@ -25,6 +25,7 @@
#include "lib/malloc/malloc.h"
#include "lib/string/compat_string.h"
+#include <stdio.h>
#include <string.h>
static void pubsub_adjmap_add(pubsub_adjmap_t *map,
@@ -343,21 +344,27 @@ lint_message(const pubsub_adjmap_t *map, message_id_t msg)
log_warn(LD_MESG|LD_BUG,
"Message \"%s\" has subscribers, but no publishers.",
get_message_id_name(msg));
+ fprintf(stderr, "SBSDEBUG: n_pub == 0 for %s\n", get_message_id_name(msg));
ok = false;
} else if (n_sub == 0) {
log_warn(LD_MESG|LD_BUG,
"Message \"%s\" has publishers, but no subscribers.",
get_message_id_name(msg));
+ fprintf(stderr, "SBSDEBUG: n_sub == 0 for %s\n", get_message_id_name(msg));
ok = false;
}
/* Check the message graph topology. */
- if (lint_message_graph(map, msg, pub, sub) < 0)
+ if (lint_message_graph(map, msg, pub, sub) < 0) {
+ fprintf(stderr, "SBSDEBUG: lint_message_graph failed for %s\n", get_message_id_name(msg));
ok = false;
+ }
/* Check whether the messages have the same fields set on them. */
- if (lint_message_consistency(msg, pub, sub) < 0)
+ if (lint_message_consistency(msg, pub, sub) < 0) {
+ fprintf(stderr, "SBSDEBUG: lint_message_consistency failed for %s\n", get_message_id_name(msg));
ok = false;
+ }
if (!ok) {
/* There was a problem -- let's log all the publishers and subscribers on
@@ -385,6 +392,7 @@ pubsub_adjmap_check(const pubsub_adjmap_t *map)
bool all_ok = true;
for (unsigned i = 0; i < map->n_msgs; ++i) {
if (lint_message(map, i) < 0) {
+ fprintf(stderr, "SBSDEBUG: lint_message failed for %u %s\n", i, get_message_id_name((message_id_t)i));
all_ok = false;
}
}
@@ -401,11 +409,15 @@ pubsub_builder_check(pubsub_builder_t *builder)
pubsub_adjmap_t *map = pubsub_build_adjacency_map(builder->items);
int rv = -1;
- if (!map)
+ if (!map) {
+ fprintf(stderr, "SBSDEBUG: pubsub_build_adjacency_map failed\n");
goto err; // should be impossible
+ }
- if (pubsub_adjmap_check(map) < 0)
+ if (pubsub_adjmap_check(map) < 0) {
+ fprintf(stderr, "SBSDEBUG: pubsub_adjmap_check failed\n");
goto err;
+ }
rv = 0;
err:
```
5. `./autogen.sh`
6. `./configure --disable-asciidoc`
7. `make`
8. `mkdir tmp`
9. `vi tmp/main.c` where `main.c` contains
```C
#include "../src/feature/api/tor_api.h"
#include <pthread.h>
#include <stdlib.h>
#include <stdio.h>
#include <unistd.h>
static void *threadMain(void *ptr) {
int *fdp = (int*)ptr;
(void)sleep(45 /* seconds */);
(void)close(*fdp);
free(fdp);
return NULL;
}
int main() {
for (;;) {
tor_main_configuration_t *config = tor_main_configuration_new();
if (config == NULL) {
exit(1);
}
char *argv[] = {
"tor",
"Log",
"notice stderr",
"DataDirectory",
"./x",
NULL,
};
int argc = 5;
if (tor_main_configuration_set_command_line(config, argc, argv) != 0) {
exit(2);
}
int filedesc = tor_main_configuration_setup_control_socket(config);
if (filedesc < 0) {
exit(3);
}
int *fdp = malloc(sizeof(*fdp));
if (fdp == NULL) {
exit(4);
}
*fdp = filedesc;
pthread_t thread;
if (pthread_create(&thread, NULL, threadMain, /* move */ fdp) != 0) {
exit(5);
}
(void)tor_run_main(config);
if (pthread_join(thread, NULL) != 0) {
exit(6);
}
fprintf(stderr, "********** doing another round\n");
}
}
```
10. `gcc -Wall tmp/main.c -L. -ltor -levent -lcrypto -lssl -lz -lm`
11. `./a.out 2>&1|tee LOG.txt`
The `tmp/main.c` command is a reasonable approximation of what our Go code for running tor does. The main difference is that we start tor with `DisableNetwork` set and re-enable network later. This difference does not seem to have any impact, since we saw aborts in both cases.
We run repeated bootstraps because the OONI Probe Android app loads tor and the Go code as a shared library and calls `tor_run_main` each time we run a OONI experiment that requires tor (typically, `vanilla_tor` and `torsf`).
### What is the current bug behavior?
We can cluster the kind of crashes we observed into two groups.
#### pubsub_adjmap_check failed
This crash has been the most frequent one we observed. With the above patch applied, it generally looks like this:
```
[... omitting logs from several bootstraps ...]
Mar 22 14:07:21.000 [notice] Owning controller connection has closed -- exiting now.
Mar 22 14:07:21.000 [notice] Catching signal TERM, exiting cleanly.
********** doing another round
SBSDEBUG: n_sub == 0 for orconn_state
SBSDEBUG: lint_message failed for 5 orconn_state
SBSDEBUG: n_pub == 0 for orconn_state
SBSDEBUG: lint_message failed for 34 orconn_state
SBSDEBUG: pubsub_adjmap_check failed
[1] 300227 IOT instruction (core dumped) ./a.out 2>&1 |
300228 done tee LOG.txt
```
When running this via Go code, we see a different message before the abort. I think this happens because Go installs its own handler for SIGABRT, while the C code does not install any handler. My understanding is also that "IOT instruction" is related to `SIGIOT`, which seems to be an alias for `SIGABRT` judging from include/linux/signal.h and Glib's bits/signum-generic.h.
My understanding of the above logs is that, somehow, a message is registered twice: once without publishers, and once without subscribers.
It's also important to point out that the message causing failure has not always been `orconn_state`. Based on all the aborts we have examined, it seems that also `orconn_status` could cause failures. For the sake of brevity, I am not going to copy here all the logs we collected, but you can read them along with my thought process when analyzing the bug at https://github.com/ooni/probe/issues/2406.
#### INTERNAL ERROR: Raw assertion failed in Tor 0.4.7.13 at src/app/main/subsysmgr.c:183: 0
This specific error occurred very rarely (2-3 times). It is not clear whether this is the same issue or not, however I think it makes sense to mention it in the same issue, because it occurred when using the above code to investigate pubsub_install aborts.
```
2023/03/21 17:59:13 info tunnel: tor: exec: <internal/libtor> x/tunnel/torsf/tor [...]
BUG: subsystem btrack (at 55) could not connect to publish/subscribe system.
============================================================ T= 1679421553
INTERNAL ERROR: Raw assertion failed in Tor 0.4.7.13 at src/app/main/subsysmgr.c:183: 0
A subsystem couldn't be connected.
./testtorsf(dump_stack_symbols_to_error_fds+0x58)[0xe6df08]
./testtorsf(tor_raw_assertion_failed_msg_+0x97)[0xe6e8d7]
./testtorsf(subsystems_add_pubsub_upto+0x128)[0xe47df8]
./testtorsf(pubsub_install+0x29)[0xdf9c99]
./testtorsf(tor_run_main+0x8a)[0xdf9e2a]
./testtorsf(_cgo_2d785783cadf_Cfunc_tor_run_main+0x1b)[0xdf665b]
./testtorsf[0x500e04]
SIGABRT: abort
PC=0x7fa00f89aa7c m=14 sigcode=18446744073709551610
signal arrived during cgo execution
```
(Because this specific error occurred when using Go code, here you see also the output of Go `SIGABRT` handler.)
The specific assertion that fails in this case is the following:
```C
int
subsystems_add_pubsub_upto(pubsub_builder_t *builder,
int target_level)
{
for (unsigned i = 0; i < n_tor_subsystems; ++i) {
const subsys_fns_t *sys = tor_subsystems[i];
if (!sys->supported)
continue;
if (sys->level > target_level)
break;
if (! sys_status[i].initialized)
continue;
int r = 0;
if (sys->add_pubsub) {
subsys_id_t sysid = get_subsys_id(sys->name);
raw_assert(sysid != ERROR_ID);
pubsub_connector_t *connector;
connector = pubsub_connector_for_subsystem(builder, sysid);
r = sys->add_pubsub(connector);
pubsub_connector_free(connector);
}
if (r < 0) {
fprintf(stderr, "BUG: subsystem %s (at %u) could not connect to "
"publish/subscribe system.", sys->name, sys->level);
raw_assert_unreached_msg("A subsystem couldn't be connected."); // <- HERE
}
}
return 0;
}
```
### What is the expected behavior?
On a very broad level, I think tor should not abort. Because I do not understand very well what is happening, it is difficult to provide a more specific recommendation about what the code should actually do.
### Environment
- Which version of Tor are you using? Run `tor --version` to get the version if you are unsure.
Always 0.4.7.13
- Which operating system are you using? For example: Debian GNU/Linux 10.1, Windows 10, Ubuntu Xenial, FreeBSD 12.2, etc.
Android (several versions and devices according to the Google Play console); Android 13 on Pixel 4a arm64 (my phone); Ubuntu 22.04.2 on amd64
- Which installation method did you use? Distribution package (apt, pkg, homebrew), from source tarball, from Git, etc.
Tor compiled along with all its dependencies using our build scripts as well as tor compiled from sources with Ubuntu 22.04.2 installation dependencies when reproducing the issue using the above mentioned steps.
### Relevant logs and/or screenshots
I think I already provided representative logs above. The https://github.com/ooni/probe/issues/2406 issue contains all the logs we produced while investigating this issue on our end. It also describes how we progressively narrowed down the problem from an abort in the Android app to an abort using Go code on Linux to the minimal instructions for reproducing the issue that I mentioned above.
On this note, I initially suspected that there was a data race on our end. That assumption was true but the abort continued to occur after I fixed the data race inside Go code. In any case, the possible presence of data races on our end prompted me to bypass our Go code and write C code that could allow reproducing the issue. In one of my final attempts at understanding the issue using just C code, I [patched tor to avoid aborting in case pubsub_install failed](https://github.com/ooni/probe/issues/2406#issuecomment-1479884981), recompiled and run with tsan enabled, [seeing just two pubsub_install failures over 490 runs and no sign of data races](https://github.com/ooni/probe/issues/2406#issuecomment-1480826748).
### Possible fixes
I don't know. Since the data-race theory is not supported by data and unlikely, perhaps it could be that state from previous runs causes issues with the pubsub subsystem that appear for repeated bootstraps? I'll be happy to collaborate and try other debugging strategies.https://gitlab.torproject.org/tpo/core/tor/-/issues/40836Update recommended/required protocol lists?2024-01-16T15:38:37ZNick MathewsonUpdate recommended/required protocol lists?We haven't updated the recommended/required protocol lists since 2021, possibly longer. If we mark more protocols as required or recommended, we can more correctly reason about the network.
The protocols that are unconditionally suppor...We haven't updated the recommended/required protocol lists since 2021, possibly longer. If we mark more protocols as required or recommended, we can more correctly reason about the network.
The protocols that are unconditionally supported by 0.4.7.7 (our oldest supported stable) are:
* `Cons=1-2 Desc=1-2 DirCache=2 FlowCtrl=1-2 HSDir=2 HSIntro=4-5 HSRend=1-2 Link=1-5 LinkAuth=3 Microdesc=1-2 Padding=2 Relay=1-4`
The protocols that are unconditionally supported by the most recent Arti are not currently listed anywhere or enforced in Arti. :disappointed: So maybe we should take care of that first?
The protocols that the consensus currently recommends are:
* `recommended-client-protocols Cons=2 Desc=2 DirCache=2 HSDir=2 HSIntro=4 HSRend=2 Link
=4-5 Microdesc=2 Relay=2`
* `recommended-relay-protocols Cons=2 Desc=2 DirCache=2 HSDir=2 HSIntro=4 HSRend=2 Link=4-5 LinkAuth=3 Microdesc=2 Relay=2`
The protocols that the consensus currently requires are:
* `required-client-protocols Cons=2 Desc=2 Link=4 Microdesc=2 Relay=2`
* `required-relay-protocols Cons=2 Desc=2 DirCache=2 HSDir=2 HSIntro=4 HSRend=2 Link=4-5 LinkAuth=3 Microdesc=2 Relay=2`
cc @dgoulet @mikeperryNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40802Dir auths say "Failed to find node for hop #2 of our path. Discarding this ci...2024-01-31T23:49:46ZRoger DingledineDir auths say "Failed to find node for hop #2 of our path. Discarding this circuit." every second after boot until new consensusStarting somewhere in Tor 0.4.7, every directory authority now prints thousands of lines of
```
Jun 01 14:51:33.790 [notice] Failed to find node for hop #2 of our path. Discarding this circuit.
```
on startup. It continues until the top ...Starting somewhere in Tor 0.4.7, every directory authority now prints thousands of lines of
```
Jun 01 14:51:33.790 [notice] Failed to find node for hop #2 of our path. Discarding this circuit.
```
on startup. It continues until the top of the hour when
```
Jun 01 14:59:59.942 [notice] Failed to find node for hop #2 of our path. Discarding this circuit.
Jun 01 15:00:00.017 [notice] Time to publish the consensus and discard old votes
Jun 01 15:00:00.162 [notice] Published ns consensus
Jun 01 15:00:00.315 [notice] Published microdesc consensus
```https://gitlab.torproject.org/tpo/core/tor/-/issues/40803Cannot write to ClientOnionAuthDir when Sandbox is enabled2023-06-05T16:46:55ZanonymCannot write to ClientOnionAuthDir when Sandbox is enabled### Summary
When `tor` has the sandbox option enabled it cannot write to the `ClientOnionAuthDir` directory to store onion auth keys, e.g. when checking the "Remember this key" checkbox in Tor Browser when providing the key.
### Steps ...### Summary
When `tor` has the sandbox option enabled it cannot write to the `ClientOnionAuthDir` directory to store onion auth keys, e.g. when checking the "Remember this key" checkbox in Tor Browser when providing the key.
### Steps to reproduce:
1. Configure `tor` with `Sandbox 1`
2. Configure `tor` with `ClientOnionAuthDir /some/writable/directory`
3. Use Tor Browser to access an onion service with onion authentication
4. Check the "Remember this key" checkbox when providing the key
### What is the current bug behavior?
The onion auth prompt in Tor Browser reports "Unable to store creds for ...", and no key is written to the `ClientOnionAuthDir` directory.
### What is the expected behavior?
No errors, and the key should be written to the `ClientOnionAuthDir` directory.
### Environment
- Tor version 0.4.7.13
- Tested both on Debian Sid and inside Tails with `tor` installed via `apt`
### Relevant logs and/or screenshots
```
Jun 02 13:04:02.000 [warn] sandbox_intern_string(): Bug: No interned sandbox parameter found for /var/lib/tor/keys/n7wwn7f4jirk2yaukobahoane722lnvi7d65emwj4toas7uf5oaomdyd.auth_private.tmp (on Tor 0.4.7.13 )
Jun 02 13:00:25.000 [warn] Couldn't open "/var/lib/tor/keys/n7wwn7f4jirk2yaukobahoane722lnvi7d65emwj4toas7uf5oaomdyd.auth_private.tmp" (/var/lib/tor/keys/n7wwn7f4jirk2yaukobahoane722lnvi7d65emwj4toas7uf5oaomdyd.auth_private) for writing: Operation not permitted
Jun 02 13:00:25.000 [warn] Failed to write client auth creds file for n7wwn7f4jirk2yaukobahoane722lnvi7d65emwj4toas7uf5oaomdyd!
```
### Possible fixes
Update the sandbox rules.Tor: 0.4.9.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40809Bring padding machines into sync with Tobias's latest changes2023-06-15T10:31:28ZMike PerryBring padding machines into sync with Tobias's latest changesTobias has added some changes to the padding machines in his latest research: in particular, the padding machine can respond to queue length/congestion signals. I believe there are now also probabilistic transitions (ie https://gitlab.to...Tobias has added some changes to the padding machines in his latest research: in particular, the padding machine can respond to queue length/congestion signals. I believe there are now also probabilistic transitions (ie https://gitlab.torproject.org/tpo/core/tor/-/issues/31636 or https://gitlab.torproject.org/tpo/core/tor/-/issues/31787).
I need to read his latest paper, sync with him, and discuss these things. This will generate new tickets.Mike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40822[warn] Possible replay detected! An INTRODUCE2 cell with the same ENCRYPTED s...2023-08-28T12:28:35ZTimeIsGold[warn] Possible replay detected! An INTRODUCE2 cell with the same ENCRYPTED section was seen 0 seconds ago. Dropping cell.I am running HS using Tor 0.4.7.13 Running on Windows Server and i am using Obfs4 Bridges for my nodes.
i got this messages every day and i don't know what is the problem
`[warn] Possible replay detected! An INTRODUCE2 cell with the sam...I am running HS using Tor 0.4.7.13 Running on Windows Server and i am using Obfs4 Bridges for my nodes.
i got this messages every day and i don't know what is the problem
`[warn] Possible replay detected! An INTRODUCE2 cell with the same ENCRYPTED section was seen 0 seconds ago. Dropping cell.`
`[notice] Your network connection speed appears to have changed. Resetting timeout to 60000ms after 18 timeouts and 1000 buildtimes.`https://gitlab.torproject.org/tpo/core/tor/-/issues/40831null pointer dereference if threadpool initialization fails2023-10-08T15:25:20ZAlex Xunull pointer dereference if threadpool initialization fails```
In function 'threadpool_register_reply_event',
inlined from 'cpuworker_init' at src/core/mainloop/cpuworker.c:140:13,
inlined from 'run_tor_main_loop' at src/app/main/main.c:1230:3,
inlined from 'tor_run_main' at src/app/...```
In function 'threadpool_register_reply_event',
inlined from 'cpuworker_init' at src/core/mainloop/cpuworker.c:140:13,
inlined from 'run_tor_main_loop' at src/app/main/main.c:1230:3,
inlined from 'tor_run_main' at src/app/main/main.c:1359:14,
inlined from 'tor_main' at src/feature/api/tor_api.c:166:12,
inlined from 'main' at src/app/main/tor_main.c:32:7:
src/lib/evloop/workqueue.c:631:9: warning: potential null pointer dereference [-Wnull-dereference]
631 | if (tp->reply_event) {
| ^
```
if `threadpool_new` fails, then `tp` will be null. `spawn_func` should not normally fail, and furthermore the result will most likely be a non-exploitable segmentation fault, but it is still technically undefined behavior and should be fixed.Tor: 0.4.9.x-freezehttps://gitlab.torproject.org/tpo/core/tor/-/issues/40840Prevent outbound cell command flipping2024-02-13T17:00:47ZMike PerryPrevent outbound cell command flippingAs per https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/344-protocol-info-leaks.txt#L197, the RELAY_EARLY fix did not address the outbound direction.
We can fix this by checking at relays that the cell command field ...As per https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/344-protocol-info-leaks.txt#L197, the RELAY_EARLY fix did not address the outbound direction.
We can fix this by checking at relays that the cell command field does not switch back and forth between RELAY and RELAY_EARLY. Then, so long as the middle relay is honest, this vector cannot be used as a covert channel between the Guard and the Exit.
This fix should be relatively simple and can be backported, though we should of course test it in shadow.Mike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40846Expired signing key2023-08-30T12:29:24ZAndreasExpired signing keyThe key referenced at https://gitlab.torproject.org/tpo/core/tor/-/blame/main/README.md#L44, and used for signing the tor-0.4.8.4 tag, is expired since Aug 23.The key referenced at https://gitlab.torproject.org/tpo/core/tor/-/blame/main/README.md#L44, and used for signing the tor-0.4.8.4 tag, is expired since Aug 23.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/40847tor 0.4.8.4: compilation error on SunOS / OpenIndiana2023-09-06T00:57:05Zsvschmeltor 0.4.8.4: compilation error on SunOS / OpenIndianaCompiling tor 0.4.8.4 throws the following error.
(Similar to https://gitlab.torproject.org/tpo/core/tor/-/issues/40843 => compilation error on NetBSD)
```
/export/home/svschmel/oi-userland/components/network/tor/tor-0.4.8.4/src/ext/equ...Compiling tor 0.4.8.4 throws the following error.
(Similar to https://gitlab.torproject.org/tpo/core/tor/-/issues/40843 => compilation error on NetBSD)
```
/export/home/svschmel/oi-userland/components/network/tor/tor-0.4.8.4/src/ext/equix/hashx/src/virtual_memory.c: In function 'hashx_vm_alloc_huge':
/export/home/svschmel/oi-userland/components/network/tor/tor-0.4.8.4/src/ext/equix/hashx/src/virtual_memory.c:113:5: error: 'MAP_HUGETLB' undeclared (first use in this function)
113 | | MAP_HUGETLB | MAP_POPULATE, -1, 0);
| ^~~~~~~~~~~
/export/home/svschmel/oi-userland/components/network/tor/tor-0.4.8.4/src/ext/equix/hashx/src/virtual_memory.c:113:5: note: each undeclared identifier is reported only once for each function it appears in
/export/home/svschmel/oi-userland/components/network/tor/tor-0.4.8.4/src/ext/equix/hashx/src/virtual_memory.c:113:19: error: 'MAP_POPULATE' undeclared (first use in this function); did you mean 'MAP_PRIVATE'?
113 | | MAP_HUGETLB | MAP_POPULATE, -1, 0);
| ^~~~~~~~~~~~
| MAP_PRIVATE
make[2]: *** [Makefile:16523: src/ext/equix/hashx/src/libhashx_a-virtual_memory.o] Error 1
make[2]: Leaving directory '/export/home/svschmel/oi-userland/components/network/tor/build/amd64'
make[1]: *** [Makefile:7648: all] Error 2
```
These defines do not exist on SunOS / OpenIndiana
=> https://www.illumos.org/man/2/mmap
I suggest handling SunOS like OpenBSD in file "virtual_memory.c" because the default => the "#else"- tree <= assumes that the underlying OS is a Linux-system.
```
...
#elif defined(__OpenBSD__)
mem = MAP_FAILED; // OpenBSD does not support huge pages
#else
mem = mmap(NULL, bytes, PAGE_READWRITE, MAP_PRIVATE | MAP_ANONYMOUS
| MAP_HUGETLB | MAP_POPULATE, -1, 0);
...
```
With this modification the code can be compiled on SunOS / Openindiana successful.https://gitlab.torproject.org/tpo/core/tor/-/issues/40901Document for the Relay Operator community how to debug relays that are slower...2023-12-19T07:53:56ZAlexander Færøyahf@torproject.orgDocument for the Relay Operator community how to debug relays that are slower than what the operator expectsThis idea origins from a conversation betweeh @beth, @gk and I on #tor-dev today.
We often release new features of C Tor to the relay operators that causes discussions/conversations around whether Tor has gotten faster/slower/uses (more...This idea origins from a conversation betweeh @beth, @gk and I on #tor-dev today.
We often release new features of C Tor to the relay operators that causes discussions/conversations around whether Tor has gotten faster/slower/uses (more|less) memory/crashes (more|less) often/etc. many of these items are hard to give a definitive "yes, the cause of this is X" and it's very time consuming for the Network Team to debug each item individually with the operator.
It would be very useful to have a document in place that informs relay operators about the different situations that may impact performance and how they can get some performance measurements they can then compare to and see if our performance have truly regressed. This can also be used to push MetricsPort to more operators.
We can expand upon the document over time as we discover new ways to do this analysis and/or from feedback from the relay operator community.
This is related to:
- https://lists.torproject.org/pipermail/tor-relays/2023-December/021409.html
- https://lists.torproject.org/pipermail/tor-relays/2023-December/021407.html
This may be relevant to Arti Relay too.
CC @mikeperry for awareness.