Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:46:45Zhttps://gitlab.torproject.org/legacy/trac/-/issues/32108tor can overrun its accountingmax if it enters soft hibernation first2020-06-13T15:46:45ZRoger Dingledinetor can overrun its accountingmax if it enters soft hibernation firstI'll put the punchline first: second_elapsed_callback(), which is where we check if it's time to go dormant for hibernation, is no longer called when we've entered soft hibernation.
I assume this is because of the new "periodic event fl...I'll put the punchline first: second_elapsed_callback(), which is where we check if it's time to go dormant for hibernation, is no longer called when we've entered soft hibernation.
I assume this is because of the new "periodic event flag" feature, where we try to avoid calling callbacks when we're in a state that won't need them. See the "online and active" note here:
```
/* This is a legacy catch-all callback that runs once per second if
* we are online and active. */
CALLBACK(second_elapsed, NET_PARTICIPANT,
FL(NEED_NET)|FL(RUN_ON_DISABLE)),
```
The impact is limited, since we stop accepting new connections and new circuits when we enter soft hibernation, but it can still be bad: existing connections and circuits can last for a long time and use a lot of bandwidth.
A secondary impact is that accounting_run_housekeeping() never gets called, which means that the state file never gets updated after we've entered soft hibernation, which means these bandwidth overspends are never recorded to disk.
I think the bug went in during commit 4bf79fa4f which is part of Tor 0.4.0.1-alpha.
The PERIODIC_EVENT_FLAG_NEED_NET flag (what FL(NEED_NET) expands into) checks net_is_disabled(), but there is another function right after net_is_disabled in netstatus.c called net_is_completely_disabled(), and the only difference is that it checks we_are_fully_hibernating() vs we_are_hibernating().
I confirmed the overall bug happens in practice: there's a relay operator in #tor who hit soft hibernation, and then saw his tor proceed to use more bytes than expected. I had him do a 'gdb attach' to his tor and break on 'second_elapsed_callback' and the function never got called.
It seems like the immediate fix, and best backport plan, would be to resume calling second_elapsed_callback even when net_is_disabled(). The longer term plan can be to audit our calls to net_is_disabled() and net_is_completely_disabled() and we_are_hibernating(), with an eye towards "should we be doing this behavior while soft hibernating", and see what other bugs we find.Tor: 0.4.0.x-finalRoger DingledineRoger Dingledinehttps://gitlab.torproject.org/legacy/trac/-/issues/31810Bug: ../src/lib/process/process_unix.c:265: process_unix_exec: Assertion line...2020-06-13T15:45:51ZTracBug: ../src/lib/process/process_unix.c:265: process_unix_exec: Assertion line should be unreached failed; aborting.I am on Debian Buster, x86_64, trying to run Tor (installed from deb.torproject.org).
```
Sep 20 00:18:33 jennis Tor[1481]: Tor 0.4.1.5 running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1c, Zlib 1.2.11, Liblzma 5.2.4, and Libzst...I am on Debian Buster, x86_64, trying to run Tor (installed from deb.torproject.org).
```
Sep 20 00:18:33 jennis Tor[1481]: Tor 0.4.1.5 running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1c, Zlib 1.2.11, Liblzma 5.2.4, and Libzstd 1.3.8.
Sep 20 00:18:33 jennis Tor[1481]: Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Sep 20 00:18:33 jennis Tor[1481]: Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Sep 20 00:18:33 jennis Tor[1481]: Read configuration file "/etc/tor/torrc".
Sep 20 00:18:33 jennis Tor[1481]: Opening Socks listener on 127.0.0.1:9050
Sep 20 00:18:33 jennis Tor[1481]: Opened Socks listener on 127.0.0.1:9050
Sep 20 00:18:33 jennis Tor[1481]: Opening DNS listener on 127.0.0.1:53
Sep 20 00:18:33 jennis Tor[1481]: Opened DNS listener on 127.0.0.1:53
Sep 20 00:18:33 jennis Tor[1481]: Opening DNS listener on [::1]:53
Sep 20 00:18:33 jennis Tor[1481]: Opened DNS listener on [::1]:53
Sep 20 00:18:33 jennis Tor[1481]: Opening Transparent pf/netfilter listener on 127.0.0.1:9040
Sep 20 00:18:33 jennis Tor[1481]: Opened Transparent pf/netfilter listener on 127.0.0.1:9040
Sep 20 00:18:33 jennis Tor[1481]: Opening Transparent pf/netfilter listener on [::1]:9040
Sep 20 00:18:33 jennis Tor[1481]: Opened Transparent pf/netfilter listener on [::1]:9040
Sep 20 00:18:33 jennis Tor[1481]: Opening Control listener on 127.0.0.1:9051
Sep 20 00:18:33 jennis Tor[1481]: Opened Control listener on 127.0.0.1:9051
Sep 20 00:18:33 jennis Tor[1481]: Parsing GEOIP IPv4 file /usr/share/tor/geoip.
Sep 20 00:18:33 jennis Tor[1482]: tor_assertion_failed_(): Bug: ../src/lib/process/process_unix.c:265: process_unix_exec: Assertion line should be unreached failed; aborting. (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: Assertion line should be unreached failed in process_unix_exec at ../src/lib/process/process_unix.c:265: . Stack trace: (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(log_backtrace_impl+0x46) [0x556988adb106] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(tor_assertion_failed_+0x147) [0x556988ad6297] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(process_unix_exec+0x274) [0x556988ab06c4] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(process_exec+0x5b) [0x556988aaea7b] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(pt_configure_remaining_proxies+0x563) [0x5569889a0093] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(set_options+0x18ba) [0x556988a5405a] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(options_init_from_string+0x382) [0x556988a55292] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(options_init_from_torrc+0x38a) [0x556988a5581a] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(tor_init+0x3c7) [0x556988927567] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(tor_run_main+0xb4) [0x556988927e74] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(tor_main+0x3a) [0x55698892638a] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(main+0x19) [0x556988925f49] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xeb) [0x7f49509c609b] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1482]: Bug: /usr/bin/tor(_start+0x2a) [0x556988925f9a] (on Tor 0.4.1.5 )
Sep 20 00:18:33 jennis Tor[1481]: Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
Sep 20 00:18:33 jennis Tor[1481]: Bootstrapped 0% (starting): Starting
Sep 20 00:18:33 jennis Tor[1481]: Starting with guard context "default"
Sep 20 00:18:33 jennis Tor[1481]: Signaled readiness to systemd
Sep 20 00:18:34 jennis Tor[1481]: Bootstrapped 5% (conn): Connecting to a relay
Sep 20 00:18:34 jennis Tor[1481]: Opening Control listener on /run/tor/control
Sep 20 00:18:34 jennis Tor[1481]: Opened Control listener on /run/tor/control
Sep 20 00:18:34 jennis Tor[1481]: Bootstrapped 10% (conn_done): Connected to a relay
Sep 20 00:18:34 jennis Tor[1481]: Bootstrapped 14% (handshake): Handshaking with a relay
Sep 20 00:18:34 jennis Tor[1481]: Bootstrapped 15% (handshake_done): Handshake with a relay done
Sep 20 00:18:34 jennis Tor[1481]: Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
Sep 20 00:18:34 jennis Tor[1481]: Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
Sep 20 00:18:34 jennis Tor[1481]: Bootstrapped 95% (circuit_create): Establishing a Tor circuit
Sep 20 00:18:35 jennis Tor[1481]: Bootstrapped 100% (done): Done
```
**Trac**:
**Username**: ParckwartTor: 0.4.0.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/31463Redundant build exclusion not working corrently on travis2020-06-13T15:44:23ZSebastian HahnRedundant build exclusion not working corrently on travisSince the hardening options are used again we're using one more builder than intendedSince the hardening options are used again we're using one more builder than intendedTor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/31091Bug stracktrace when pluggable transport cannot bind to port2020-06-13T15:43:26Zs7rBug stracktrace when pluggable transport cannot bind to portI have just setup some new obfs4 fast bridges running latest obfs4proxy from yawning / master (locally compiled on my server) and Tor 0.4.2.0-alpha-dev and encountered a bug stack trace when started to try with the pluggable transport li...I have just setup some new obfs4 fast bridges running latest obfs4proxy from yawning / master (locally compiled on my server) and Tor 0.4.2.0-alpha-dev and encountered a bug stack trace when started to try with the pluggable transport listening on a port < 1024. This is well known, which is why even in the wiki page it is recommended and properly documented to use setcap in order to grant permission to the PT executable to bind to lower ports, but shouldn't it just warn and exit, instead of offering all this:
```
Jul 06 05:51:18.000 [warn] Server managed proxy encountered a method error. (obfs4 listen tcp <ipv4>:<port>: bind: permission denied)
Jul 06 05:51:18.000 [warn] Managed proxy at '/usr/local/bin/obfs4proxy' failed the configuration protocol and will be destroyed.
Jul 06 05:51:18.000 [err] tor_assertion_failed_(): Bug: ../src/feature/client/transports.c:1836: managed_proxy_stdout_callback: Assertion mp->conf_state == PT_PROTO_COMPLETED failed; aborting. (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: Assertion mp->conf_state == PT_PROTO_COMPLETED failed in managed_proxy_stdout_callback at ../src/feature/client/transports.c:1836: . Stack trace: (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(log_backtrace_impl+0x47) [0x559a576f5d87] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(tor_assertion_failed_+0x147) [0x559a576f0ed7] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(+0xd4a89) [0x559a575b8a89] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(+0x1e4e4b) [0x559a576c8e4b] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/lib/x86_64-linux-gnu/libevent-2.0.so.5(event_base_loop+0x6a0) [0x7f6f343265a0] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(do_main_loop+0x105) [0x559a57555ed5] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(tor_run_main+0x1245) [0x559a57543905] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(tor_main+0x3a) [0x559a57540cfa] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(main+0x19) [0x559a57540879] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf1) [0x7f6f32b7b2e1] (on Tor 0.4.2.0-alpha-dev )
Jul 06 05:51:18.000 [err] Bug: /usr/bin/tor(_start+0x2a) [0x559a575408ca] (on Tor 0.4.2.0-alpha-dev )
```
What if it just stopped here:
```
Jul 06 05:51:18.000 [warn] Server managed proxy encountered a method error. (obfs4 listen tcp <ip>:<port>: bind: permission denied)
Jul 06 05:51:18.000 [warn] Managed proxy at '/usr/local/bin/obfs4proxy' failed the configuration protocol and will be destroyed.
```
Or maybe even exit entirely so the operator can know something is really wrong.
Also, I don't know if this is related or not, I will try to make more tests to confirm or infirm this, but under Debian stable the bridges that are not run with setcap don't get a reasonable measured speed. Those with setcap that listen to lower ports have a bw of 2.5 - 3.5 MiB/s and those without setcap have a bw of 12 KiB/s - 70 KiB/s and they are all on the same infrastructure / hardware resources / internet speed. I will do some more digging to confirm this, right now 2 out of 2 obfs4 bridges that were configured without setcap did not get good bandwidth measurement even after 15 days of continuous uptime.Tor: 0.4.0.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/31003heap-use-after-free src/feature/nodelist/routerlist.c:704 in router_get_by_de...2020-06-13T15:43:07ZDavid Gouletdgoulet@torproject.orgheap-use-after-free src/feature/nodelist/routerlist.c:704 in router_get_by_descriptor_digestDoing some HS DoS testing and on ctrl+c of my tor client (unmodified), this showed up.
Tor version 0.4.2.0-alpha-dev (git-6afe1b00c9c73b1b).
(info.log attached to the ticket)
```
==16279==ERROR: AddressSanitizer: heap-use-after-free o...Doing some HS DoS testing and on ctrl+c of my tor client (unmodified), this showed up.
Tor version 0.4.2.0-alpha-dev (git-6afe1b00c9c73b1b).
(info.log attached to the ticket)
```
==16279==ERROR: AddressSanitizer: heap-use-after-free on address 0x60e000002428 at pc 0x559683ab9839 bp 0x7ffff3007db0 sp 0x7ffff3007da0
READ of size 8 at 0x60e000002428 thread T0
#0 0x559683ab9838 in router_get_by_descriptor_digest src/feature/nodelist/routerlist.c:704
#1 0x559683aa2a12 in count_usable_descriptors src/feature/nodelist/nodelist.c:2388
#2 0x559683aa2f75 in compute_frac_paths_available src/feature/nodelist/nodelist.c:2448
#3 0x559683aaf204 in update_router_have_minimum_dir_info src/feature/nodelist/nodelist.c:2701
#4 0x559683aaf204 in router_have_minimum_dir_info src/feature/nodelist/nodelist.c:2301
#5 0x559683a52714 in can_client_refetch_desc src/feature/hs/hs_client.c:1184
#6 0x559683a52714 in hs_client_refetch_hsdesc src/feature/hs/hs_client.c:1350
#7 0x559683a56bc2 in retry_all_socks_conn_waiting_for_desc src/feature/hs/hs_client.c:298
#8 0x559683a56bc2 in hs_client_dir_info_changed src/feature/hs/hs_client.c:1936
#9 0x559683abab62 in routerlist_free_ src/feature/nodelist/routerlist.c:944
#10 0x559683abab62 in routerlist_free_all src/feature/nodelist/routerlist.c:1429
#11 0x5596838ce3f4 in tor_free_all src/app/main/shutdown.c:116
#12 0x5596838cc0c4 in tor_run_main src/app/main/main.c:1358
#13 0x5596838c86b8 in tor_main src/feature/api/tor_api.c:164
#14 0x5596838c1dbf in main src/app/main/tor_main.c:32
#15 0x7f6565a75b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)
#16 0x5596838c7db9 in _start (/home/dgoulet/Documents/git/tor/src/app/tor+0x1ccdb9)
0x60e000002428 is located 8 bytes inside of 160-byte region [0x60e000002420,0x60e0000024c0)
freed by thread T0 here:
#0 0x7f656659f75f in __interceptor_free (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10d75f)
#1 0x559683ab6fa4 in routerlist_free_ src/feature/nodelist/routerlist.c:968
#2 0x559683abab62 in routerlist_free_ src/feature/nodelist/routerlist.c:944
#3 0x559683abab62 in routerlist_free_all src/feature/nodelist/routerlist.c:1429
#4 0x5596838ce3f4 in tor_free_all src/app/main/shutdown.c:116
#5 0x5596838cc0c4 in tor_run_main src/app/main/main.c:1358
#6 0x5596838c86b8 in tor_main src/feature/api/tor_api.c:164
#7 0x5596838c1dbf in main src/app/main/tor_main.c:32
#8 0x7f6565a75b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)
previously allocated by thread T0 here:
#0 0x7f656659fb58 in malloc (/usr/lib/x86_64-linux-gnu/libasan.so.5+0x10db58)
#1 0x559683c7804e in tor_malloc_ src/lib/malloc/malloc.c:45
#2 0x559683c780e3 in tor_malloc_zero_ src/lib/malloc/malloc.c:71
#3 0x559683ab99f1 in router_get_routerlist src/feature/nodelist/routerlist.c:812
#4 0x559683aa4a88 in nodelist_assert_ok src/feature/nodelist/nodelist.c:853
#5 0x559683aace28 in nodelist_set_consensus src/feature/nodelist/nodelist.c:662
#6 0x559683a9b54a in networkstatus_set_current_consensus src/feature/nodelist/networkstatus.c:2137
#7 0x559683a9beb9 in reload_consensus_from_file src/feature/nodelist/networkstatus.c:1761
#8 0x559683a9bf8c in router_reload_consensus_networkstatus src/feature/nodelist/networkstatus.c:278
#9 0x5596838cb17f in run_tor_main_loop src/app/main/main.c:1180
#10 0x5596838cc0b4 in tor_run_main src/app/main/main.c:1328
#11 0x5596838c86b8 in tor_main src/feature/api/tor_api.c:164
#12 0x5596838c1dbf in main src/app/main/tor_main.c:32
#13 0x7f6565a75b6a in __libc_start_main (/lib/x86_64-linux-gnu/libc.so.6+0x26b6a)
SUMMARY: AddressSanitizer: heap-use-after-free src/feature/nodelist/routerlist.c:704 in router_get_by_descriptor_digest
Shadow bytes around the buggy address:
0x0c1c7fff8430: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
0x0c1c7fff8440: 00 00 00 02 fa fa fa fa fa fa fa fa 00 00 00 00
0x0c1c7fff8450: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 05
0x0c1c7fff8460: fa fa fa fa fa fa fa fa 00 00 00 00 00 00 00 00
0x0c1c7fff8470: 00 00 00 00 00 00 00 00 00 00 06 fa fa fa fa fa
=>0x0c1c7fff8480: fa fa fa fa fd[fd]fd fd fd fd fd fd fd fd fd fd
0x0c1c7fff8490: fd fd fd fd fd fd fd fd fa fa fa fa fa fa fa fa
0x0c1c7fff84a0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c1c7fff84b0: fd fd fd fa fa fa fa fa fa fa fa fa fd fd fd fd
0x0c1c7fff84c0: fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd fd
0x0c1c7fff84d0: fa fa fa fa fa fa fa fa fd fd fd fd fd fd fd fd
Shadow byte legend (one shadow byte represents 8 application bytes):
Addressable: 00
Partially addressable: 01 02 03 04 05 06 07
Heap left redzone: fa
Freed heap region: fd
Stack left redzone: f1
Stack mid redzone: f2
Stack right redzone: f3
Stack after return: f5
Stack use after scope: f8
Global redzone: f9
Global init order: f6
Poisoned by user: f7
Container overflow: fc
Array cookie: ac
Intra object redzone: bb
ASan internal: fe
Left alloca redzone: ca
Right alloca redzone: cb
Shadow gap: cc
```Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/31001Undefined behavior in tor_vasprintf()2020-06-13T15:43:04ZGeorge KadianakisUndefined behavior in tor_vasprintf()```
Overflowing a signed integer in C is an undefined behaviour.
It is possible to trigger this undefined behaviour in tor_asprintf on
Windows or systems lacking vasprintf.
On these systems, eiter _vscprintf or vsnprintf is called to re...```
Overflowing a signed integer in C is an undefined behaviour.
It is possible to trigger this undefined behaviour in tor_asprintf on
Windows or systems lacking vasprintf.
On these systems, eiter _vscprintf or vsnprintf is called to retrieve
the required amount of bytes to hold the string. These functions can
return INT_MAX. The easiest way to recreate this is the use of a
specially crafted configuration file, e.g. containing the line:
FirewallPorts AAAAA<in total 2147483610 As>
This line triggers the needed tor_asprintf call which eventually
leads to an INT_MAX return value from _vscprintf or vsnprintf.
The needed byte for \0 is added to the result, triggering the
overflow and therefore the undefined behaviour.
Casting the value to size_t before addition fixes the behaviour.
```Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/30894Memory leak on invalid CSV_INTERVAL value2020-06-13T15:42:38ZNick MathewsonMemory leak on invalid CSV_INTERVAL valueThere is a missing free in the case where we fail to parse the interval from a CSV_INTERVAL field.There is a missing free in the case where we fail to parse the interval from a CSV_INTERVAL field.Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/30871circuit_mark_for_close_(): Bug: Duplicate call to circuit_mark_for_close at ....2020-06-13T15:42:34Zs7rcircuit_mark_for_close_(): Bug: Duplicate call to circuit_mark_for_close at ../src/feature/hs/hs_service.c:3166 (first at ../src/feature/hs/hs_service.c:2385)I experienced this today after normally turning on a machine that was off for physical movement. OS is Debian and Tor is set with systemctl to start on boot, so it's a normal out of the box boring setup. Internet was 100% working on the ...I experienced this today after normally turning on a machine that was off for physical movement. OS is Debian and Tor is set with systemctl to start on boot, so it's a normal out of the box boring setup. Internet was 100% working on the machine while it was booting up, and of course cable and correct network settings properly setup before powered on.
The Tor instance hosts 2 onion services, one v2 and one v3.
```
un 12 11:05:50.000 [notice] Tor 0.4.1.2-alpha-dev opening log file.
Jun 12 11:05:50.086 [notice] Tor 0.4.1.2-alpha-dev running on Linux with Libevent 2.0.21-stable, OpenSSL 1.1.0j, Zlib 1.2.8, Liblzma 5.2.2, and Libzstd 1.1.2.
Jun 12 11:05:50.086 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jun 12 11:05:50.086 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Jun 12 11:05:50.086 [notice] Read configuration file "/usr/share/tor/tor-service-defaults-torrc".
Jun 12 11:05:50.086 [notice] Read configuration file "/etc/tor/torrc".
Jun 12 11:05:50.089 [notice] Opening Socks listener on 127.0.0.1:9050
Jun 12 11:05:50.089 [notice] Opened Socks listener on 127.0.0.1:9050
Jun 12 11:05:50.000 [notice] Parsing GEOIP IPv4 file /usr/share/tor/geoip.
Jun 12 11:05:50.000 [notice] Parsing GEOIP IPv6 file /usr/share/tor/geoip6.
Jun 12 11:05:50.000 [notice] Bootstrapped 0% (starting): Starting
Jun 12 11:05:50.000 [notice] Starting with guard context "default"
Jun 12 11:05:50.000 [notice] Signaled readiness to systemd
Jun 12 11:05:51.000 [notice] Opening Control listener on /run/tor/control
Jun 12 11:05:51.000 [notice] Opened Control listener on /run/tor/control
Jun 12 11:05:51.000 [notice] Bootstrapped 5% (conn): Connecting to a relay
Jun 12 11:05:52.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 1; recommendation warn; host E9423220AAE845EE4B693A7C4235787C05D56280 at 185.117.82.23:9001)
Jun 12 11:05:52.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 2; recommendation warn; host 6B9EA8927AB6E94E216067E65372182343A5AFA2 at 62.210.83.207:8080)
Jun 12 11:05:52.000 [warn] 1 connections have failed:
Jun 12 11:05:52.000 [warn] 1 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:05:55.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 3; recommendation warn; host 96CAA917F65BCD62CECD236F67652BFD7C69E52E at 104.37.193.102:443)
Jun 12 11:05:55.000 [warn] 2 connections have failed:
Jun 12 11:05:55.000 [warn] 2 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:05:55.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 4; recommendation warn; host 73C62311F5650010A4D32E09F141A0B574EFCAF5 at 51.38.140.87:9001)
Jun 12 11:05:55.000 [warn] 3 connections have failed:
Jun 12 11:05:55.000 [warn] 3 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:05:58.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 5; recommendation warn; host 2E23B75B9D9AB7B86D1D5AB1C9B6B30BA0E84E3E at 148.251.51.66:9001)
Jun 12 11:05:58.000 [warn] 4 connections have failed:
Jun 12 11:05:58.000 [warn] 4 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:05:58.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 6; recommendation warn; host 7BE683E65D48141321C5ED92F075C55364AC7123 at 193.23.244.244:443)
Jun 12 11:05:58.000 [warn] 5 connections have failed:
Jun 12 11:05:58.000 [warn] 5 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:06:01.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 7; recommendation warn; host 268216C455A8322E07733961A29110110958D1BF at 73.211.181.17:9001)
Jun 12 11:06:01.000 [warn] 6 connections have failed:
Jun 12 11:06:01.000 [warn] 6 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:06:06.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 8; recommendation warn; host 7EA6EAD6FD83083C538F44038BBFA077587DD755 at 194.109.206.212:443)
Jun 12 11:06:06.000 [warn] 7 connections have failed:
Jun 12 11:06:06.000 [warn] 7 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:06:09.000 [warn] Problem bootstrapping. Stuck at 5% (conn): Connecting to a relay. (No route to host; NOROUTE; count 9; recommendation warn; host FD2F9B881AC640100C428DF47DC9A863DC2F2536 at 37.187.17.67:9001)
Jun 12 11:06:09.000 [warn] 8 connections have failed:
Jun 12 11:06:09.000 [warn] 8 connections died in state connect()ing with SSL state (No SSL object)
Jun 12 11:06:13.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
Jun 12 11:06:13.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
Jun 12 11:06:13.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
Jun 12 11:06:13.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
Jun 12 11:06:13.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
Jun 12 11:06:13.000 [notice] Bootstrapped 30% (loading_status): Loading networkstatus consensus
Jun 12 11:06:14.000 [notice] Bootstrapped 75% (enough_dirinfo): Loaded enough directory info to build circuits
Jun 12 11:06:15.000 [notice] Bootstrapped 90% (ap_handshake_done): Handshake finished with a relay to build circuits
Jun 12 11:06:15.000 [notice] Bootstrapped 95% (circuit_create): Establishing a Tor circuit
Jun 12 11:06:16.000 [notice] Bootstrapped 100% (done): Done
Jun 12 11:06:21.000 [warn] Hidden service <XXX> exceeded launch limit with 10 intro points in the last 31 seconds. Intro circuit launches are limited to 10 per 300 seconds.
Jun 12 11:06:21.000 [warn] Service configured in "/var/lib/tor/<V2 service was here>":
Jun 12 11:06:21.000 [warn] Intro point 0 at [scrubbed]: circuit is open
Jun 12 11:06:21.000 [warn] Intro point 1 at [scrubbed]: circuit is open
Jun 12 11:06:21.000 [warn] Intro point 2 at [scrubbed]: circuit is waiting to see how other guards perform
Jun 12 11:06:21.000 [warn] Intro point 3 at [scrubbed]: circuit is waiting to see how other guards perform
Jun 12 11:06:21.000 [warn] Intro point 4 at [scrubbed]: circuit is waiting to see how other guards perform
Jun 12 11:11:19.000 [warn] Unknown introduction point auth key on circuit 3339485114 for service [scrubbed]
Jun 12 11:11:19.000 [warn] circuit_mark_for_close_(): Bug: Duplicate call to circuit_mark_for_close at ../src/feature/hs/hs_service.c:3166 (first at ../src/feature/hs/hs_service.c:2385) (on Tor 0.4.1.2-alpha-dev )
Jun 12 11:11:34.000 [warn] Unknown introduction point auth key on circuit 3672980732 for service [scrubbed]
Jun 12 11:11:34.000 [warn] circuit_mark_for_close_(): Bug: Duplicate call to circuit_mark_for_close at ../src/feature/hs/hs_service.c:3166 (first at ../src/feature/hs/hs_service.c:2385) (on Tor 0.4.1.2-alpha-dev )
Jun 12 12:44:47.000 [warn] Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Jun 12 12:44:49.000 [warn] Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Jun 12 12:44:49.000 [notice] Closing stream for '[scrubbed].onion': hidden service is unavailable (try again later).
```
The machine was off for more than 24 hours. It was simply shut down normally while Tor was running previously using sudo poweroff. The consensus it had on disk when it booted wasn't up to date any more of course. Somehow it wanted to connect to the guards first, but could not, even it had nothing blocking it. This is why it reported the bootstrapping problem and NOROUTE, when it Bootstrapped 100% Done, these messages stopped. Either there is something broken in how Tor starts up in some scenarios, either the network interface was not raised by the OS until after Tor service started, and then it was able to bootstrapp? I have tried to ping and traceroute all the IP addresses of the Guards that were printed in the log, and they were all available and reachable to me.
The most concerning message is **circuit_mark_for_close_(): Bug: Duplicate call to circuit_mark_for_close at ../src/feature/hs/hs_service.c:3166 (first at ../src/feature/hs/hs_service.c:2385) (on Tor 0.4.1.2-alpha-dev )** that needs fix.
Also: Unknown introduction point auth key on circuit 3339485114 for service [scrubbed] this is very interesting. I have absolutely no auth set up for neither the v3 neither the v2 hidden service.Tor: 0.4.0.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/30649Every few hours, relays [warn] Received circuit padding stop command for unkn...2020-06-13T15:41:53ZteorEvery few hours, relays [warn] Received circuit padding stop command for unknown machine.toralf says on #tor-relays:
```
I got 5x within last 36 ours with 0.4.1.1-alpha: [warn] Received circuit padding stop command for unknown machine. -shall I ignore that?
```
I'm marking this as 041-must, because we will get lots ...toralf says on #tor-relays:
```
I got 5x within last 36 ours with 0.4.1.1-alpha: [warn] Received circuit padding stop command for unknown machine. -shall I ignore that?
```
I'm marking this as 041-must, because we will get lots of bug reports from relay operators if we release a stable with this warning. At the very least, we must downgrade it to a protocol error.Tor: 0.4.0.x-finalMike PerryMike Perryhttps://gitlab.torproject.org/legacy/trac/-/issues/30614Use MAP_INHERIT_NONE/ZERO if available instead of crashing on assertion failure2020-06-13T15:41:42ZriastradhUse MAP_INHERIT_NONE/ZERO if available instead of crashing on assertion failureOn NetBSD, the way to ask a page range be zero'd or unmapped on fork is `minherit(addr, nbytes, MAP_INHERIT_ZERO)` or `minherit(addr, nbytes, MAP_INHERIT_NONE)`. However,
- map_anon.c does not actually check for or use `MAP_INHERIT_ZER...On NetBSD, the way to ask a page range be zero'd or unmapped on fork is `minherit(addr, nbytes, MAP_INHERIT_ZERO)` or `minherit(addr, nbytes, MAP_INHERIT_NONE)`. However,
- map_anon.c does not actually check for or use `MAP_INHERIT_ZERO` or `MAP_INHERIT_NONE`, so
- `FLAG_NOINHERIT` is undefined, so
- `tor_mmap_anonymous` fails to disinherit the mapping, so
- `crypto_fast_rng_new_from_seed` throws a fit.
```
slow/prob_distr/stochastic_log_logistic: [forking] May 25 03:56:58.091 [err] tor_assertion_failed_(): Bug: src/lib/crypt_ops/crypto_rand_fast.c:184: crypto_fast_rng_new_from_seed: Assertion inherit != INHERIT_RES_KEEP failed; aborting. (on Tor 0.4.1.1-alpha-dev 29955f13e5bc8e61)
May 25 03:56:58.091 [err] Bug: Assertion inherit != INHERIT_RES_KEEP failed in crypto_fast_rng_new_from_seed at src/lib/crypt_ops/crypto_rand_fast.c:184: . (Stack trace not available) (on Tor 0.4.1.1-alpha-dev 29955f13e5bc8e61)
[Lost connection!]
```
The attached patch checks for and uses `MAP_INHERIT_ZERO` and `MAP_INHERIT_NONE`. **However, it does not adjust the code path that gets confused if `tor_mmap_anonymous` fails to disinherit the mapping,** which might be worth doing too, perhaps earlier on by detecting and noisily reporting a lack of support for cutting off the hereditary concentration of wealth, or at least secrets, in society before it's too late.Tor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/30263make shellcheck expects scripts in the build directory2020-06-13T15:40:56Zteormake shellcheck expects scripts in the build directoryIn #28058, we added shellcheck for coverage and cov-diff. But we expect coverage to be in the build directory, which is wrong.
This fails the newest version of shellcheck in out-of-tree builds.In #28058, we added shellcheck for coverage and cov-diff. But we expect coverage to be in the build directory, which is wrong.
This fails the newest version of shellcheck in out-of-tree builds.Tor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/30109Document that MapAddress is automatically strict, but does not handle redirects2020-06-13T15:40:32ZteorDocument that MapAddress is automatically strict, but does not handle redirectsWe should document that:
1. StrictNodes does not apply to MapAddress
2. MapAddress ~~falls back to a random exit by default~~ fails rather than falling back to a random exit
Edited to add:
3. If the site does a redirect, MapAddress does...We should document that:
1. StrictNodes does not apply to MapAddress
2. MapAddress ~~falls back to a random exit by default~~ fails rather than falling back to a random exit
Edited to add:
3. If the site does a redirect, MapAddress does not apply to the new siteTor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/30004when reloading a consensus with a CRLF, log at notice.2020-06-13T15:40:13ZNick Mathewsonwhen reloading a consensus with a CRLF, log at notice.Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/30001test failure: dir_handle_get/status_vote_next_bandwidth2020-06-13T15:40:12ZNick Mathewsontest failure: dir_handle_get/status_vote_next_bandwidthFound by travis: https://travis-ci.org/torproject/tor/jobs/515199996
```
dir_handle_get/status_vote_next_bandwidth:
FAIL src/test/test_dir_handle_get.c:2535: assert(strstr(header, expires))
[status_vote_next_bandwidth FAILED]
```
...Found by travis: https://travis-ci.org/torproject/tor/jobs/515199996
```
dir_handle_get/status_vote_next_bandwidth:
FAIL src/test/test_dir_handle_get.c:2535: assert(strstr(header, expires))
[status_vote_next_bandwidth FAILED]
```
To me this looks like a race condition: the clock probably has advanced by a second between the call to directory_handle_command_get() and the call to time(NULL).Tor: 0.4.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/29961Update the Tor version for bandwidth-file-digest in torspec2020-06-13T15:40:01ZteorUpdate the Tor version for bandwidth-file-digest in torspecSee my pull request:
https://github.com/torproject/torspec/pull/73See my pull request:
https://github.com/torproject/torspec/pull/73Tor: 0.4.0.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/29959Actually include the bandwidth file digest in the vote2020-06-13T15:40:00ZteorActually include the bandwidth file digest in the voteThe original code in #26698 had a logic error, so the bandwidth file digest was never included in the vote.The original code in #26698 had a logic error, so the bandwidth file digest was never included in the vote.Tor: 0.4.0.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/29897Refactor handle_get_next_bandwidth() to use connection_dir_buf_add()2020-06-13T15:39:50ZteorRefactor handle_get_next_bandwidth() to use connection_dir_buf_add()After #21377 merges, we want to refactor handle_get_next_bandwidth() to use connection_dir_buf_add().
If we backport this refactor to 0.4.0 or earlier, we'll also need #29896 for the connection_dir_buf_add() backport.After #21377 merges, we want to refactor handle_get_next_bandwidth() to use connection_dir_buf_add().
If we backport this refactor to 0.4.0 or earlier, we'll also need #29896 for the connection_dir_buf_add() backport.Tor: 0.4.0.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/29874torrc no longer accepts space in executable paths2020-06-13T15:39:40Zcypherpunkstorrc no longer accepts space in executable pathsI'm not sure exactly when this change happened but 0.3.5.8 still works with a space in the executable path while 0.4.0.2 fails to launch the executable with the same torrc file.
The following is an example torrc line that no longer work...I'm not sure exactly when this change happened but 0.3.5.8 still works with a space in the executable path while 0.4.0.2 fails to launch the executable with the same torrc file.
The following is an example torrc line that no longer works:
ClientTransportPlugin obfs2,obfs3,obfs4,scramblesuit exec C:\Program Files\obfs4proxy.exe
Is this change documented somewhere? I'd like to keep the executable path unchanged if there is a simple workaroundTor: 0.4.0.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/29706Test failure due to memory leaks in shared-random unit tests: long-term fix2020-06-13T15:39:11ZteorTest failure due to memory leaks in shared-random unit tests: long-term fixIn #29599 we fixed some leaks in the shared-random unit tests. But there are still some test failures.
The shared random state claims to take over ownership of SRVs passed to it using PUT. But it doesn't free them automatically: instead...In #29599 we fixed some leaks in the shared-random unit tests. But there are still some test failures.
The shared random state claims to take over ownership of SRVs passed to it using PUT. But it doesn't free them automatically: instead, the caller has to remember to call state_del_current_srv() first. (Or one of its callers.)
The current app code is ok, but the test code doesn't always call the functions in the right order.Tor: 0.4.0.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/29693Decrease probability of stochastic failures in test-slow2020-06-13T15:39:05ZteorDecrease probability of stochastic failures in test-slowOur stochastic tests are supposed to fail around 1 in 100 runs. But when I'm doing a backport to 0.2.9, there are up to 14 jobs times 9 branches, each of which runs a test instance.
So let's decrease the probability to about 1 in (100 *...Our stochastic tests are supposed to fail around 1 in 100 runs. But when I'm doing a backport to 0.2.9, there are up to 14 jobs times 9 branches, each of which runs a test instance.
So let's decrease the probability to about 1 in (100 * 14 * 9).
Here's what the output looks like:
```
slow/prob_distr/stochastic_uniform: [forking] fail uniform sampler
FAIL src/test/test_prob_distr.c:1209: assert(ok)
NOTE: This is a stochastic test, and we expect it to fail from
time to time, with some low probability. If you see it fail more
than one trial in 100, though, please tell us.
Seed: 5DB9A3B32C29B76D7A0032700DD142BB
[stochastic_uniform FAILED]
```
https://travis-ci.org/torproject/tor/jobs/503432646#L5845Tor: 0.4.0.x-finalGeorge KadianakisGeorge Kadianakis