Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:53:12Zhttps://gitlab.torproject.org/legacy/trac/-/issues/33929Tor relay appears offline in Metrics after some times2020-06-13T15:53:12ZTracTor relay appears offline in Metrics after some timesSince at least December 20, my relay appears offline in Metrics after few hours to few days (typically between 3h to 3d). In nyx, all flags disappears. After restarting the service, flags are coming back after few hours and the relay app...Since at least December 20, my relay appears offline in Metrics after few hours to few days (typically between 3h to 3d). In nyx, all flags disappears. After restarting the service, flags are coming back after few hours and the relay appears online again. There is nothing in the log.
My fingerprint:
33D88F331408141F2A2CC563239E54E48F7A211B
Notice that this happens to at least another person, its fingerprint
DAA806E9529D77EF94685FC0E513386EF65B83F8
**Trac**:
**Username**: clementhttps://gitlab.torproject.org/legacy/trac/-/issues/33925Can't compile Tor master as of 2020-04-162020-06-13T15:53:12ZTracCan't compile Tor master as of 2020-04-16I can compile 0.4.3.4-rc without problem now.
Master not so great, though:
```
Ld
/Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
atzjucxhwnhfrqcmzwlymuguursh/Build/Products/Debug/Tor.framework/Versions
/A/Tor normal x86_64 (in...I can compile 0.4.3.4-rc without problem now.
Master not so great, though:
```
Ld
/Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
atzjucxhwnhfrqcmzwlymuguursh/Build/Products/Debug/Tor.framework/Versions
/A/Tor normal x86_64 (in target 'Tor-Mac' from project 'Tor') cd
/Users/berhart/workspace/gp/Tor.framework
/Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.
/xctoolchain/usr/bin/clang -target x86_64-apple-macos10.9
/-dynamiclib -isysroot
//Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.
/platform/Developer/SDKs/MacOSX10.15.sdk
/-L/Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
/atzjucxhwnhfrqcmzwlymuguursh/Build/Products/Debug
/-F/Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
/atzjucxhwnhfrqcmzwlymuguursh/Build/Products/Debug -filelist
//Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
/atzjucxhwnhfrqcmzwlymuguursh/Build/Intermediates.noindex/Tor.build/
/Debug/Tor-Mac.build/Objects-normal/x86_64/Tor.LinkFileList
/-install_name @rpath/Tor.framework/Tor -Xlinker -rpath -Xlinker
/@executable_path/../Frameworks -Xlinker -rpath -Xlinker
/@loader_path/../Frameworks -Xlinker -object_path_lto -Xlinker
//Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
/atzjucxhwnhfrqcmzwlymuguursh/Build/Intermediates.noindex/Tor.build/
/Debug/Tor-Mac.build/Objects-normal/x86_64/Tor_lto.o -Xlinker
/-export_dynamic -Xlinker -no_deduplicate -fobjc-arc
/-fobjc-link-runtime -fapplication-extension -framework CFNetwork
/-ltor-app -ltor-compress -ltor-evloop -ltor-tls -ltor-crypt-ops
/-lkeccak-tiny -ltor-pubsub -ltor-confmgt -led25519_ref10
/-led25519_donna -lcurve25519_donna -ltor-geoip -ltor-process
/-ltor-buf -ltor-time -ltor-fs -ltor-encoding -ltor-sandbox
/-ltor-container -ltor-net -ltor-thread -ltor-memarea -ltor-math
/-ltor-meminfo -ltor-osinfo -ltor-dispatch -ltor-log -ltor-lock
/-ltor-fdio -ltor-string -ltor-term -ltor-smartlist-core
/-ltor-malloc -ltor-wallclock -ltor-err -ltor-version -ltor-intmath
/-ltor-ctime -lor-trunnel -ltor-trace -lssl -lcrypto -levent
/-levent_core -levent_extra -levent_pthreads -llzma -lz -Xlinker
/-dependency_info -Xlinker
//Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
/atzjucxhwnhfrqcmzwlymuguursh/Build/Intermediates.noindex/Tor.build/
/Debug/Tor-Mac.build/Objects-normal/x86_64/Tor_dependency_info.dat
/-o
//Users/berhart/Library/Developer/Xcode/DerivedData/Tor-
/atzjucxhwnhfrqcmzwlymuguursh/Build/Products/Debug/Tor.framework/
/Versions/A/Tor
/
Undefined symbols for architecture x86_64: "_sys_winprocess", referenced
from: _tor_subsystems in libtor-app.a(subsystem_list.o) ld: symbol(s)
not found for architecture x86_64 clang: error: linker command failed
with exit code 1 (use -v to see invocation)
```
See #33837 for more context.
**Trac**:
**Username**: tlaTor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33924Tor beginning with 0.4.1 seems to ignore SIGNAL SHUTDOWN on iOS2020-06-13T15:53:11ZTracTor beginning with 0.4.1 seems to ignore SIGNAL SHUTDOWN on iOSOk, so it seems Tor doesn't honor SIGNAL SHUTDOWN anymore. The first Tor version I tested was 0.4.1.5 (first stable of 0.4.1).
Sending it with 0.4.0.6, I see this in the logs:
```
Apr 16 16:31:58.000 [notice] Interrupt: exiting cleanly....Ok, so it seems Tor doesn't honor SIGNAL SHUTDOWN anymore. The first Tor version I tested was 0.4.1.5 (first stable of 0.4.1).
Sending it with 0.4.0.6, I see this in the logs:
```
Apr 16 16:31:58.000 [notice] Interrupt: exiting cleanly.
```
With later Tor versions, this message disappears and the TorThread still hangs around after app wake up.
**Trac**:
**Username**: tlaTor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33919Write unit tests for channel_matches_target_addr_for_extend()2020-06-13T15:53:10ZteorWrite unit tests for channel_matches_target_addr_for_extend()In #33817, we modify channel_matches_target_addr_for_extend() to take an IPv4 and an IPv6 address.
We also modify channel_get_for_extend() to take an IPv4 and an IPv6 address. It has existing tests, but they ignore IP addresses. (And mo...In #33817, we modify channel_matches_target_addr_for_extend() to take an IPv4 and an IPv6 address.
We also modify channel_get_for_extend() to take an IPv4 and an IPv6 address. It has existing tests, but they ignore IP addresses. (And mock the matches_target() method, which is called by channel_matches_target_addr_for_extend().)
It would be useful to have tests that check that a channel matches, if its IP address matches the IPv4 or IPv6 address. But it's not urgent. (And the actual changes to the function are pretty trivial.)
The setup for these tasks is a bit tricky, we need to set up a channel_tls_t and an associated connection_t with a real_addr. (Note that #33898 may change the name of real_addr.)
This task is not urgent or important. Anyone can do this task.Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33918Stop truncating IPv6 addresses in channel logs2020-06-13T15:53:10ZteorStop truncating IPv6 addresses in channel logschannel_tls_get_remote_desc_method() formats IP addresses and ports, but its buffer is only 32 characters. IPv6 addresses can be up to 41 characters long, and the port is an extra 6 characters.channel_tls_get_remote_desc_method() formats IP addresses and ports, but its buffer is only 32 characters. IPv6 addresses can be up to 41 characters long, and the port is an extra 6 characters.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33917Make IF_BUG_ONCE() support ALL_BUGS_ARE_FATAL2020-06-13T15:53:09ZteorMake IF_BUG_ONCE() support ALL_BUGS_ARE_FATAL... and DISABLE_ASSERTS_IN_UNIT_TESTS.
It's the only assertion/bug macro that doesn't support these debugging modes. Let's fix that.... and DISABLE_ASSERTS_IN_UNIT_TESTS.
It's the only assertion/bug macro that doesn't support these debugging modes. Let's fix that.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33905Adjust "large number of connections to other relays" warnings2020-06-13T15:53:08ZteorAdjust "large number of connections to other relays" warningsIn #33880, we adjusted the limits MIN_RELAY_CONNECTIONS_TO_WARN, and the default number of connections allowed per relay.
In #33048, we will allow relays to extend via IPv6 as well as IPv4. So we should make these changes:
* warn if any...In #33880, we adjusted the limits MIN_RELAY_CONNECTIONS_TO_WARN, and the default number of connections allowed per relay.
In #33048, we will allow relays to extend via IPv6 as well as IPv4. So we should make these changes:
* warn if any relay has more than 4 connections (that is, more than 2 sides multiplied by 2 IP addresses, if there is disagreement over canonical connections).
* ignore 2 connections per relay.
Clients should only extend between two relays that support the Relay=3 protocol version. So we shouldn't need to backport this change.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33901Allow IPv6 extend cells2020-06-13T15:53:08ZteorAllow IPv6 extend cellsAs part of #33817, we want to allow IPv6 extend cells.
We want clients and relays to send:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cells
We want relays to parse:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cellsAs part of #33817, we want to allow IPv6 extend cells.
We want clients and relays to send:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cells
We want relays to parse:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cellsTor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33900Check for invalid zero IPv4 addresses and ports in extend cells2020-06-13T15:53:07ZteorCheck for invalid zero IPv4 addresses and ports in extend cellsWhen we send and parse extend cells, we check that their IPv4 address field is not AF_UNSPEC.
But we should also check for zero IPv4 addresses and zero ports. (Which are both invalid.)
Code and points in #33817, I just needed a separat...When we send and parse extend cells, we check that their IPv4 address field is not AF_UNSPEC.
But we should also check for zero IPv4 addresses and zero ports. (Which are both invalid.)
Code and points in #33817, I just needed a separate bug number.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33899Allow IPv6 addresses to be canonical2020-06-13T15:53:06ZteorAllow IPv6 addresses to be canonicalIn #17604, we allowed IPv6 connections to set is_canonical_to_peer. But we did not let them set is_canonical.
So channel_tls_process_netinfo_cell() handles IPv6 addresses, but connection_or_check_canonicity() does not.
This inconsisten...In #17604, we allowed IPv6 connections to set is_canonical_to_peer. But we did not let them set is_canonical.
So channel_tls_process_netinfo_cell() handles IPv6 addresses, but connection_or_check_canonicity() does not.
This inconsistency could be one cause of the logging issues in #24841.
We should allow IPv6 connections to be canonical. We should also do some cleanup of the related connection address logging in #33898.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33898Stop modifying addr on connections, and delete real_addr2020-06-13T15:53:06ZteorStop modifying addr on connections, and delete real_addrIn connection_or_check_canonicity(), we overwrite conn.addr with the canonical address of the relay in the consensus. That makes accurate logging impossible.
And so we also need channel.real_addr, to store the actual address of the remo...In connection_or_check_canonicity(), we overwrite conn.addr with the canonical address of the relay in the consensus. That makes accurate logging impossible.
And so we also need channel.real_addr, to store the actual address of the remote peer. And for some reason, we also have conn.address, a string copy of the peer's canonical address and port.
See:
https://github.com/torproject/tor/blob/7f9eaec538b7d01e0d1b130dc4cf2ec634252d46/src/core/or/connection_or.c#L920
This is... a mess.
Here's the high-level interface I'd like to see:
* use a function to format a connection or channel addresses for loogging
* use exactly as many address and port variables as needed in connection and channel (no extras!)
* qualify each address and port variable's name with its purpose
For example, here's one possible design:
* delete addr, port, address, and real_addr
* add remote_ap, a tor_addr_port_t that is the remote address and port of the TCP connection to the remote peer
* implement connection_describe(), which:
* formats remote_ap,
* formats is_canonical (and any other useful info), and
* calls node_describe() to format the canonical IPv4 and IPv6 addresses and ports of the remote peer.
We may also need separate fields for reverse proxied addresses, see the comment at:
https://github.com/torproject/tor/blob/7517e1b5d31aada1f594c2594737a231d9d8e116/src/core/or/channeltls.c#L1339
If we need separate variables or functions for channels, we can use a similar design. (But ideally, re-using as many functions and variables as possible.)
This is important for Sponsor 55: getting accurate connection information will make diagnosing bugs much easier.Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33894make (retroactive) proposal for DoS subsystem2020-06-13T15:53:04ZRoger Dingledinemake (retroactive) proposal for DoS subsystemIn #24902, dgoulet speaks of a ddos-design.txt document.
But there is no actual proposal for the overall DoS subsystem.
If we have the document around, and we just never published it, this is a great chance to notice, clean it up a bit...In #24902, dgoulet speaks of a ddos-design.txt document.
But there is no actual proposal for the overall DoS subsystem.
If we have the document around, and we just never published it, this is a great chance to notice, clean it up a bit, and call it proposal three-hundred-and-something. (And then maybe turn some of it into one of the spec files if that makes sense, but, one step at a time here. :)
Motivated by this month's tor-dev thread where all we have to show for the DoS subsystem design is a trac ticket number and a changelog entry.Tor: unspecifiedDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33889Functions with char* arguments are dangerous when used with casting2020-06-13T15:53:04ZGeorge KadianakisFunctions with char* arguments are dangerous when used with castingThere are various functions in our codebase which receive char* arguments when they actually receive a byte array. That's I guess because of the old C standard when uint8_t didn't really exist, so we had to use char* for everything.
Sti...There are various functions in our codebase which receive char* arguments when they actually receive a byte array. That's I guess because of the old C standard when uint8_t didn't really exist, so we had to use char* for everything.
Still, these days we use uint8_t* and this means that we need to cast it to char* when we use those functions. This can cause security issues because this casting is silencing type-safety warnings (as an example see https://github.com/torproject/tor/pull/1843#discussion_r406191281 from #33545).
For example, see how `fast_mem_is_zero()` is used 75% of the time with casting its first argument. Instead of doing this, we could make a new `fast_mem_is_zero_uint8t()` function that takes uint8_t as argument. Or even make a `fast_mem_is_zero_char()` function that takes char*.
In other functions, it might be possible to replace the `char*` with `void*` but in the case of `fast_mem_is_zero()` that's not possible because of the implementation of the function.
Other potential problem functions: `base64_encode()` `hex_str()` `crypto_rand()` `crypto_digest()`.Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33883Fix typo in router_build_fresh_unsigned_routerinfo() comment2020-06-13T15:53:03ZNeel Chauhanneel@neelc.orgFix typo in router_build_fresh_unsigned_routerinfo() commentThis comment in `router_build_fresh_unsigned_routerinfo()`:
```
/* If there is no valud IPv6 ORPort, the address and port are null. */
```
should be:
```
/* If there is no valid IPv6 ORPort, the address and port are null. */
```This comment in `router_build_fresh_unsigned_routerinfo()`:
```
/* If there is no valud IPv6 ORPort, the address and port are null. */
```
should be:
```
/* If there is no valid IPv6 ORPort, the address and port are null. */
```Tor: 0.4.4.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33880Confusing "Your relay has a very large number of connections to other relays"...2020-06-13T15:53:02ZRoger DingledineConfusing "Your relay has a very large number of connections to other relays" relay messageA new relay operator reports this complaint in their logs, showing up hourly:
```
Your relay has a very large number of connections to other relays. Is your
outbound address the same as your relay address? Found 13 connections to 8
relay...A new relay operator reports this complaint in their logs, showing up hourly:
```
Your relay has a very large number of connections to other relays. Is your
outbound address the same as your relay address? Found 13 connections to 8
relays. Found 13 current canonical connections, in 0 of which we were a
non-canonical peer. 5 relays had more than 1 connection, 0 had more than 2, and
0 had more than 4 connections.
```
I checked, and their outbound address was the same as their relay address.
Upon investigation, it looks like the redundant connections are to directory authorities.
My theory is that the change from #17592 (which went into 0.3.1.1-alpha, commit d5a151a0) is responsible: while before that canonical relay-to-relay connections would expire after either side first reached its randomized "15 to 22.5 minute" timeout, now they expire after either side reaches its "45 to 75 minute" timeout. And since directory authorities test reachability every 1280 seconds (around 21.3 minutes), that means it is expected that most relays will have duplicate canonical connections with directory authorities.
Possible fixes:
(A) Change the notice-level log to make it clearer that it's not scary, or at least it's not actionable. Maybe that means making it info-level so nobody will see it. Probably not the best option, assuming there *are* cases where we do want relay operators to hear it.
(B) In channel_check_for_duplicates(), change `MIN_RELAY_CONNECTIONS_TO_WARN 5` to a high enough number that even if we have 2 canonical conns per authority, plus a bit more, the log message still doesn't trigger.
(C) In channel_check_for_duplicates(), skip over connections to directory authorities in some way, since we know they will be special.
(D) Make connections to or from directory authorities expire quicker, on the theory that they don't really need the same level of padding protection as other connections.
(E) Your idea here?
I'd be fine with any of B,C,D. Whichever one can be done with an easy, short, and non-invasive patch is my favorite. Maybe that's "B, raise it to 30"? That would make the message trigger when we have connections to more than 30 relays and also we have more than 45 connections open. Or we could pick the more conservative "raise it to 40", on the theory that small numbers are more likely to have edge cases and less indicative of major network problems anyway.
And while we're at it, it might be smart to say in the log message what action we want the relay operator take, e.g. "Please report:".Tor: unspecifiedNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33873client: Send back SOCKS extended error F6 in case of bad hostname2020-06-13T15:53:01ZDavid Gouletdgoulet@torproject.orgclient: Send back SOCKS extended error F6 in case of bad hostnameWith the new awesome TB 9.5a11, the SocksPort `ExtendedErrors` are being handled.
Passing an invalid v3 address that is of the right length in bytes, will send back the `F6` error but not for an invalid address like: `asijdijasdoijqwoie...With the new awesome TB 9.5a11, the SocksPort `ExtendedErrors` are being handled.
Passing an invalid v3 address that is of the right length in bytes, will send back the `F6` error but not for an invalid address like: `asijdijasdoijqwoieqw.onion`.
Problem is that we only send back the `F6` code if the address was identified, by length, as a v3. We should handle the `BAD_HOSTNAME` error code from `parse_extended_hostname()` and thus send back that extended error.Tor: 0.4.4.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33861vanguards: circ_max_megabytes applied to all connection2020-06-13T15:53:00Zcypherpunksvanguards: circ_max_megabytes applied to all connection```
# This means that applications that require large data submission (eg
# SecureDrop or onionshare) should set this much higher
# (or set to 0 to disable):
circ_max_megabytes = 8
```
My site is less than 4MB so above config is okay.
...```
# This means that applications that require large data submission (eg
# SecureDrop or onionshare) should set this much higher
# (or set to 0 to disable):
circ_max_megabytes = 8
```
My site is less than 4MB so above config is okay.
I thought vanguards only applies this limit to:
1. My onion service <--- Tor user (incoming)
2. My onion service ---> Tor user (outgoing)
However your vanguards is breaking other connections such as:
1. apt with Tor[1]
2. wget download over Tor to clearnet site
3. curl POST something over Tor to clearnet site
Problem 1. I don't want to stop vanguards just for apt and other thing.
Problem 2. I don't want to increase circ_max value just for this.
So could you please add a switch to limit only my-onion-site related connection and ignore else?
say,
```
# If true, vanguards will not apply max_mega limit non-onion connections.
# If false(default) vanguards will apply max_mega limit to all Tor connections.
# If your circ_max_megabytes is already 0, this settings does nothing.
circ_max_mega_ignore_clearnet_destination = true
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33860Finish test_onionskin_answer()2020-06-13T15:53:00ZteorFinish test_onionskin_answer()In #33633, we finished unit tests for test_extend() and helpers.
But we didn't have time to finish test_onionskin_answer().
Let's try to test each of the cases of each `if` statement in onionskin_answer(). It's ok to mock the functions...In #33633, we finished unit tests for test_extend() and helpers.
But we didn't have time to finish test_onionskin_answer().
Let's try to test each of the cases of each `if` statement in onionskin_answer(). It's ok to mock the functions that are called by onionskin_answer().Tor: unspecifiedNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33850log rotation for /var/log/tor/debug.log did not close handle to old file afte...2020-06-13T15:52:59ZTraclog rotation for /var/log/tor/debug.log did not close handle to old file after compressionso / ran full after a week or so. I found the culprits with:
find /proc/*/fd -ls | grep '(deleted)'
Version: tor 0.4.2.7-1~d8.jessie+1
Config:
Standard torrc with one Hidden service, using an HTTPSProxy and notice+debug logging enabl...so / ran full after a week or so. I found the culprits with:
find /proc/*/fd -ls | grep '(deleted)'
Version: tor 0.4.2.7-1~d8.jessie+1
Config:
Standard torrc with one Hidden service, using an HTTPSProxy and notice+debug logging enabled:
Log notice file /var/log/tor/notices.log
Log debug file /var/log/tor/debug.log
**Trac**:
**Username**: MaKoTorTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33844Do next iteration of proposal by folding in comments from dgoulet/mike2020-06-13T15:52:58ZGeorge KadianakisDo next iteration of proposal by folding in comments from dgoulet/mikeThis is just a ticket to track progress: making the next iteration of the proposal based on dgoulet/mike comments is the next step here.This is just a ticket to track progress: making the next iteration of the proposal based on dgoulet/mike comments is the next step here.Tor: unspecifiedGeorge KadianakisGeorge Kadianakis