Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T17:36:11Zhttps://gitlab.torproject.org/legacy/trac/-/issues/23508large clock skews cause numerous bootstrap UX issues2020-06-13T17:36:11ZTaylor Yularge clock skews cause numerous bootstrap UX issuesSetting the system clock several hours ahead or behind real time can cause many different bootstrapping problems. Often bootstrapping will get stuck with no obvious way to make progress, and no visible indication of what might actually ...Setting the system clock several hours ahead or behind real time can cause many different bootstrapping problems. Often bootstrapping will get stuck with no obvious way to make progress, and no visible indication of what might actually be wrong. These seem to be lead to a disproportionate number of support requests.
Some examples are:
clock in past:
* stuck at 40% (Loading authority key certs)
clock in future:
* stuck at 80% (Connecting to the Tor network)
* stuck at 85% (Finishing handshake with first hop)
More details and child tickets as I investigate further.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/23459prop224: Specialize interface of hs_circuitmap_get_rend_circ_client_side()2020-06-13T15:13:28ZGeorge Kadianakisprop224: Specialize interface of hs_circuitmap_get_rend_circ_client_side()We currently use `hs_circuitmap_get_rend_circ_client_side()` for two reasons:
a) To proceed with the rend protocol as a client when we receive an intro ack (in `handle_introduce_ack_success()`).
b) To close useless rend circuits in `clos...We currently use `hs_circuitmap_get_rend_circ_client_side()` for two reasons:
a) To proceed with the rend protocol as a client when we receive an intro ack (in `handle_introduce_ack_success()`).
b) To close useless rend circuits in `close_or_reextend_intro_circ()`.
To fit these two scenarios, the function `hs_circuitmap_get_rend_circ_client_side()` currently returns all sorts of rend circs (established and unestablished).
We can improve the logic and semantics here by splitting into two funcs. One that returns only established circs (used for (a)), and another that retuns all kinds of circs (used for (b)).Tor: 0.3.3.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/23163Wrong name for new tor config options2020-06-13T15:12:37ZDamian JohnsonWrong name for new tor config optionsHi Nick, tor has three new config options where the double unscore should be at the start rather than end...
```
SchedulerHighWaterMark__ 101 MB ...Hi Nick, tor has three new config options where the double unscore should be at the start rather than end...
```
SchedulerHighWaterMark__ 101 MB
SchedulerLowWaterMark__ 100 MB
SchedulerMaxFlushCells__ 1000
```
Lacking this causes applications (like Nyx) to present these as user-facing options rather than a hidden option like `__ReloadTorrcOnSIGHUP`.
Cheers! -DamianTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/23126HSDirs should publish some count about new-style onion addresses2020-06-13T18:09:12ZRoger DingledineHSDirs should publish some count about new-style onion addressesRight now we have an ongoing estimate of the total number of onion addresses published to the HSDirs:
https://metrics.torproject.org/hidserv-dir-onions-seen.html
How many of those are 224-style onion addresses, and how many of them are ...Right now we have an ongoing estimate of the total number of onion addresses published to the HSDirs:
https://metrics.torproject.org/hidserv-dir-onions-seen.html
How many of those are 224-style onion addresses, and how many of them are legacy-style onion addresses?
I see a `rep_hist_stored_maybe_new_hs()` for the v2-style descriptors, and I think I see a
```
/* XXX: Update HS statistics. We should have specific stats for v3. */
```
for the v3-style descriptors.
So I think that means that the graph is only showing v2-style onions, and we have no infrastructure for noticing trends with v3 style onions.
I also suspect that noticing trends is harder with v3-style onions, since each descriptor the hsdir sees is standalone, and it's not possible (without knowing the onion address) to link two descriptors to the same address.
So our only chance at estimating total number of v3 onion addresses is to know the publishing habits of v3 style onion services (how many descriptors per how much time period), and then publish the total number of descriptors we see, and folks can do some math afterwards to estimate number of running services? In any case we can see if the number goes up or down over time.
Or maybe there is some even better design? :)
The reason I bring it up now is (a) if we want to get any code into relays, we need to do it sufficiently before we need it, so it can get rolled out, and (b) I see discussions about bugs with v3-style onion services publishing every 2 minutes, and while we're fixing those we should keep in mind how handy it would be to be able to predict how many descriptors a new onion service will publish per time period on average.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/23113Manage DNS state better when "All nameservers have failed"2020-06-13T15:12:26ZteorManage DNS state better when "All nameservers have failed"We should downgrade this warning when it only happens for a short period of time (or a small number of requests), or when it happens in response to a malformed request.
This warning is causing operators to make sub-optimal DNS server ch...We should downgrade this warning when it only happens for a short period of time (or a small number of requests), or when it happens in response to a malformed request.
This warning is causing operators to make sub-optimal DNS server choices: for example, avoiding using a local cache in favour of remote resolvers.
Sometimes changing the local resolver makes a difference:
https://trac.torproject.org/projects/tor/ticket/1936#comment:12
Sometimes it happens in response to malformed requests:
https://trac.torproject.org/projects/tor/ticket/11600#comment:6
Sometimes it's harmless:
https://trac.torproject.org/projects/tor/ticket/11600#comment:7
Because it's followed by:
```
[notice] eventdns: Nameserver <ISP-resolver2>:53 is back up
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/23107prop224: Optimize hs_circ_service_get_intro_circ() digest calculation2020-06-13T15:12:24ZGeorge Kadianakisprop224: Optimize hs_circ_service_get_intro_circ() digest calculationOur prop224 function for getting an intro circ given its intro object re-calculates the relay identity fpr all the time:
```
if (ip->base.is_only_legacy) {
uint8_t digest[DIGEST_LEN];
if (BUG(crypto_pk_get_digest(ip->legacy_key...Our prop224 function for getting an intro circ given its intro object re-calculates the relay identity fpr all the time:
```
if (ip->base.is_only_legacy) {
uint8_t digest[DIGEST_LEN];
if (BUG(crypto_pk_get_digest(ip->legacy_key, (char *) digest) < 0)) {
goto end;
}
circ = hs_circuitmap_get_intro_circ_v2_service_side(digest);
```
We could shove that in the `hs_service_intro_point_t` object as well to cut some digest calculations.Tor: 0.3.4.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/23094prop224: Optimize hsdir_index calculation2020-06-13T15:25:09ZGeorge Kadianakisprop224: Optimize hsdir_index calculationLet's include `hsdir_index_t` in the node_t instead of a pointer that we allocate, since that structure is always present anyway.
See here for more details:
https://oniongit.eu/dgoulet/tor/merge_requests/6#note_412Let's include `hsdir_index_t` in the node_t instead of a pointer that we allocate, since that structure is always present anyway.
See here for more details:
https://oniongit.eu/dgoulet/tor/merge_requests/6#note_412Tor: 0.3.4.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/23082tor_addr_parse is overly permissive2020-06-13T15:42:01ZDavid Fifielddcf@torproject.orgtor_addr_parse is overly permissivetor_addr_parse allows these surprising address formats:
* `[192.0.2.1]` (IPv4 in square brackets) → 192.0.2.1
* `[11.22.33.44` (IPv4 with left square bracket only) → 11.22.33.4
* `[11:22::33:44` (IPv6 with left square bracket only) → ...tor_addr_parse allows these surprising address formats:
* `[192.0.2.1]` (IPv4 in square brackets) → 192.0.2.1
* `[11.22.33.44` (IPv4 with left square bracket only) → 11.22.33.4
* `[11:22::33:44` (IPv6 with left square bracket only) → 11:22::33:4
* `11:22::33:44:` (IPv6 with trailing colon) → 11:22::33:44Tor: 0.4.0.x-finalrl1987rl1987https://gitlab.torproject.org/legacy/trac/-/issues/22994Use consistently named constants for relay command specifications2020-06-13T15:11:59ZteorUse consistently named constants for relay command specificationsThe original spec uses RELAY_*:
https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n1257
The onion service specs and code use RELAY_COMMAND_*:
https://gitweb.torproject.org/torspec.git/tree/rend-spec.txt#n102
https://gitweb.tor...The original spec uses RELAY_*:
https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt#n1257
The onion service specs and code use RELAY_COMMAND_*:
https://gitweb.torproject.org/torspec.git/tree/rend-spec.txt#n102
https://gitweb.torproject.org/torspec.git/tree/proposals/224-rend-spec-ng.txt#n252
https://gitweb.torproject.org/tor.git/tree/src/or/or.h#n562
We should probably make these consistent. Eventually.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22936Apply Windows Tor build patch for VS20152020-06-13T15:11:48ZteorApply Windows Tor build patch for VS2015There's a patch for building Tor in Visual Studio 2015 at:
https://github.com/wbenny/vs-tor/blob/master/vs2015/tor/tor_win32_patch.diff
We should work out if we can apply it (there's no licence), or if we can learn useful things from it...There's a patch for building Tor in Visual Studio 2015 at:
https://github.com/wbenny/vs-tor/blob/master/vs2015/tor/tor_win32_patch.diff
We should work out if we can apply it (there's no licence), or if we can learn useful things from it to improve our Windows support.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22907Investigate using cargo-vendor for offline dependencies2020-06-13T15:11:39ZIsis LovecruftInvestigate using cargo-vendor for offline dependenciesPeople on the cargo team recommended we look into using https://github.com/alexcrichton/cargo-vendor to facilitate our offline builds. (See also #22830)People on the cargo team recommended we look into using https://github.com/alexcrichton/cargo-vendor to facilitate our offline builds. (See also #22830)Tor: 0.3.4.x-finalIsis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/legacy/trac/-/issues/22889Add "no peer authentication" to the list of client identification methods in ...2020-06-13T15:11:31ZteorAdd "no peer authentication" to the list of client identification methods in tor-specCREATE_FAST is no longer a reliable client discriminator:
```
If an OR sees a circuit created with CREATE_FAST, the OR is sure to be the
first hop of a circuit. ORs SHOULD reject attempts to create streams with
RELAY_BEGIN exit...CREATE_FAST is no longer a reliable client discriminator:
```
If an OR sees a circuit created with CREATE_FAST, the OR is sure to be the
first hop of a circuit. ORs SHOULD reject attempts to create streams with
RELAY_BEGIN exiting the circuit at the first hop: letting Tor be used as a
single hop proxy makes exit nodes a more attractive target for compromise.
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22881Unreachable relays launch multiple testing circuits per second2020-06-13T15:11:29ZteorUnreachable relays launch multiple testing circuits per secondWhen I start a relay behind a NAT with:
```
tor ORPort 12345 PublishServerDescriptor 0 Log "info stderr"
```
I see it launch multiple testing circuits every second.
This places a lot of load on the network: it should use exponential bac...When I start a relay behind a NAT with:
```
tor ORPort 12345 PublishServerDescriptor 0 Log "info stderr"
```
I see it launch multiple testing circuits every second.
This places a lot of load on the network: it should use exponential backoff instead.
We wanted to fix things like this as part of #4580 or #8625, but it seems we missed this case. Maybe we need to do exponential backoff at a lower level than directory requests?
Here are the log messages for one second:
```
Jul 11 19:07:20.000 [info] circuit_finish_handshake: Finished building circuit hop:
Jul 11 19:07:20.000 [info] internal circ (length 3, last hop Unnamed): $3E13E2EB87CCF5690564EE33E9F9F9F80B229FBB(open) $34D2E3C853C6C60A5CCD93E363CD524BA962D3D6(open) $4345CA43B1F4E1C08271F8FA907B4AE890591111(closed)
Jul 11 19:07:20.000 [info] circuit_send_next_onion_skin: Sending extend relay cell.
Jul 11 19:07:20.000 [info] consider_testing_reachability: Testing reachability of my ORPort: 124.171.199.202:12345.
Jul 11 19:07:20.000 [info] onion_pick_cpath_exit: Using requested exit node '$4345CA43B1F4E1C08271F8FA907B4AE890591111~Unnamed at 124.171.199.202'
Jul 11 19:07:20.000 [info] extend_info_from_node: Not including the ed25519 ID for $7202A9449EC084A305EB19A2A2A0E245FA8C111E~jonasBebe2 at 193.200.241.195, since it won't be able to authenticate it.
Jul 11 19:07:20.000 [info] extend_info_from_node: Not including the ed25519 ID for $CA618B8FB8FDF1C670BA7139B1E3EFCE25771551~bentleywashere3 at 198.199.90.205, since it won't be able to authenticate it.
Jul 11 19:07:20.000 [info] circuit_handle_first_hop: Next router is [scrubbed]: Not connected. Connecting.
Jul 11 19:07:20.000 [info] connection_or_set_identity_digest: Set identity digest for 0x7fcefb6f1cd0 ([scrubbed]): 7202A9449EC084A305EB19A2A2A0E245FA8C111E <unset>.
Jul 11 19:07:20.000 [info] connection_or_set_identity_digest: (Previously: 0000000000000000000000000000000000000000 <unset>)
Jul 11 19:07:20.000 [info] channel_tls_process_versions_cell: Negotiated version 4 with [scrubbed]:443; Sending cells: CERTS
Jul 11 19:07:20.000 [info] connection_or_client_learned_peer_id: learned peer id for 0x7fcefaf87090 ([scrubbed]): 650ABAB06345EEC29BB0B5322C27DC6E5E888019, <null>
Jul 11 19:07:20.000 [info] channel_tls_process_certs_cell: Got some good certificates from [scrubbed]:443: Authenticated it with RSA
Jul 11 19:07:20.000 [info] channel_tls_process_auth_challenge_cell: Got an AUTH_CHALLENGE cell from [scrubbed]:443: Sending authentication type 1
Jul 11 19:07:20.000 [info] circuit_send_next_onion_skin: First hop: finished sending CREATE cell to '$650ABAB06345EEC29BB0B5322C27DC6E5E888019~supc0re at 217.160.15.198'
Jul 11 19:07:20.000 [info] channel_tls_process_netinfo_cell: Got good NETINFO cell from [scrubbed]:443; OR connection is now open, using protocol version 4. Its ID digest is 650ABAB06345EEC29BB0B5322C27DC6E5E888019. Our address is apparently 124.171.199.202.
Jul 11 19:07:20.000 [info] circuit_expire_building: Abandoning circ 198 37.187.3.106:443:3622949943 (state 0,0:doing handshakes, purpose 18, len 3)
Jul 11 19:07:20.000 [info] internal circ (length 3, last hop Unnamed): $597422ADDADAF5FB2727369B7EFC7AA76B89D613(open) $E7578F3484C8ABD5FB3130E31F30C167206BCA7E(open) $4345CA43B1F4E1C08271F8FA907B4AE890591111(waiting for keys)
Jul 11 19:07:20.000 [info] circuit_testing_failed: Our testing circuit (to see if your ORPort is reachable) has failed. I'll try again later.
Jul 11 19:07:20.000 [info] circuit_finish_handshake: Finished building circuit hop:
Jul 11 19:07:20.000 [info] internal circ (length 3, last hop Unnamed): $650ABAB06345EEC29BB0B5322C27DC6E5E888019(open) $45D6A70CAEC4A2C2FCE46A127E461FF852A41280(closed) $4345CA43B1F4E1C08271F8FA907B4AE890591111(closed)
Jul 11 19:07:20.000 [info] circuit_send_next_onion_skin: Sending extend relay cell.
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22848Increase default Tor buffer sizes on Windows2020-06-13T15:11:24ZteorIncrease default Tor buffer sizes on WindowsIn #22798, a Windows relay operator discovered that setting
```
ConstrainedSockSize 262144
```
made inbound connections to their relay much faster. (Outbound connections are fast regardless.)
They suspect this is because Tor's networkin...In #22798, a Windows relay operator discovered that setting
```
ConstrainedSockSize 262144
```
made inbound connections to their relay much faster. (Outbound connections are fast regardless.)
They suspect this is because Tor's networking calls turn Windows buffer auto-tuning off.
We should improve this default, because it will help Windows relays be faster. (It shouldn't affect Windows clients, because they only connect out.)
This might be an issue in Tor, or it might be in mingw-64.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22825test: Add unit tests for circuit_send_next_onion_skin()2020-06-13T15:11:17ZDavid Gouletdgoulet@torproject.orgtest: Add unit tests for circuit_send_next_onion_skin()This is a core function that has just been refactored with #22804 and is modified by #22810.
Whatever happens, we _need_ unit tests for this as we have none now.This is a core function that has just been refactored with #22804 and is modified by #22810.
Whatever happens, we _need_ unit tests for this as we have none now.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22758When starting with no authority certificates, launch certificate downloads in...2020-06-13T15:10:53ZNick MathewsonWhen starting with no authority certificates, launch certificate downloads in parallel with consensus downloadWe could save a little time, I bet, if we fetched "all certificates" from the relevant guard at the same time we were bootstrapping our first consensus.We could save a little time, I bet, if we fetched "all certificates" from the relevant guard at the same time we were bootstrapping our first consensus.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22663Tor keeps using a very slow guard2020-06-13T15:10:21ZteorTor keeps using a very slow guardI'm running Tor Browser 7.0.1 with tor 0.3.0.8.
My guard slowed down significantly in the past week.
It became very hard to get pages to load completely, although sometimes partial data was returned.
I expect that after a lot of failur...I'm running Tor Browser 7.0.1 with tor 0.3.0.8.
My guard slowed down significantly in the past week.
It became very hard to get pages to load completely, although sometimes partial data was returned.
I expect that after a lot of failures to connect to an exit (there are hundreds in my logs) tor would at least try another guard.
But it didn't, and there is no easy way for me to tell it to try another guard. (New Circuit only changes middle and exit, the state file isn't reloaded on HUP #22662, and I didn't try New Identity, but NEWNYM is not documented to reset guards).
Maybe making it hard to change guards is a feature, but if it is, we need to either use more than 1 guard (linkability implications), or give up on a slow guard.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22660Guard against stack smashing attacks in tor with additional compiler options.2020-06-13T15:10:20ZteorGuard against stack smashing attacks in tor with additional compiler options.If we tor with -fstack-check (GCC, it's a no-op in clang[0]), it will protect against stack smashing attacks that jump the stack guard page(s):
```
Recompile all userland code (ld.so, libraries, binaries) with GCC's
"-fstack-check" opt...If we tor with -fstack-check (GCC, it's a no-op in clang[0]), it will protect against stack smashing attacks that jump the stack guard page(s):
```
Recompile all userland code (ld.so, libraries, binaries) with GCC's
"-fstack-check" option, which prevents the stack-pointer from moving
into another memory region without accessing the stack guard-page (it
writes one word to every 4KB page allocated on the stack).
```
III Solutions, https://www.qualys.com/2017/06/19/stack-clash/stack-clash.txt
"
-fstack-check
Generate code to verify that you do not go beyond the boundary of the stack. You should specify this flag if you are running in an environment with multiple threads, but only rarely need to specify it in a single-threaded environment since stack overflow is automatically detected on nearly all systems if there is only one stack.
Note that this switch does not actually cause checking to be done; the operating system must do that. The switch causes generation of code to ensure that the operating system sees the stack being extended.
"
http://gcc.gnu.org/onlinedocs/gcc-4.3.6/gcc/Code-Gen-Options.html#Code-Gen-Options
This protects against:
```
- a local-root exploit against ld.so and most SUID-root binaries
(CVE-2017-1000366, CVE-2017-1000379) on amd64 Debian, Ubuntu, Fedora,
CentOS;
```
There are remote attack possibilities mentioned in the paper as well.
We might also want to add:
```
-Wl,-z,noexecstack and -Wl,-z,noexecheap
```
https://www.owasp.org/index.php/C-Based_Toolchain_Hardening#GCC.2FBinutils
[0]: for clsng, we could use -fsanitize=safe-stack instead, but that's more intrusive: https://blog.quarkslab.com/clang-hardening-cheat-sheet.htmlTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22641If you setconf socksport=1000, it returns 553 error but then it sets it anywa...2020-06-13T15:10:16ZRoger DingledineIf you setconf socksport=1000, it returns 553 error but then it sets it anyway, and getconf socksport still returns 9050 even though that's closedRun your Tor not as root, then connect to your control port and do `setconf socksport=1000`. Tor will reply with
```
553 Unable to set option: Failed to bind one of the listener ports.
```
and it will log
```
Jun 17 15:48:09.939 [notice]...Run your Tor not as root, then connect to your control port and do `setconf socksport=1000`. Tor will reply with
```
553 Unable to set option: Failed to bind one of the listener ports.
```
and it will log
```
Jun 17 15:48:09.939 [notice] Opening Socks listener on 127.0.0.1:1000
Jun 17 15:48:09.939 [warn] Could not bind to 127.0.0.1:1000: Permission denied
Jun 17 15:48:09.939 [notice] Closing no-longer-configured Socks listener on 127.0.0.1:9050
Jun 17 15:48:09.939 [warn] Controller gave us config lines that didn't validate: Failed to bind one of the listener ports.
```
So it closed my socksport 9050, and then it was unhappy with the 1000.
Soon, I get another log line:
```
Jun 17 15:48:33.592 [notice] Opening Socks listener on 127.0.0.1:1000
Jun 17 15:48:33.592 [warn] Could not bind to 127.0.0.1:1000: Permission denied
Jun 17 15:48:33.592 [notice] Closing no-longer-configured Socks listener on 127.0.0.1:9050
```
Looks like Tor tries periodically to open it, in hopes that it'll work this time.
Bug one is that Tor shouldn't reply to the setconf with an error but then secretly honor it anyway.
And woah, check this out:
```
getconf socksport
250 SocksPort=9050
```
So bug number two is that it looks like Tor has been unsynchronized about what its SocksPort actually is.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22619SessionGroup = Reading config failed2020-06-13T15:10:12ZTracSessionGroup = Reading config failedIf i specify SessionGroup as described in manual. tor stops with error.
torrc:
SocksPort 9051 SessionGroup=1
Jun 15 16:46:24.700 [warn] Invalid SocksPort option '"SessionGroup=INT"'
Jun 15 16:46:24.700 [warn] Failed to parse/validate...If i specify SessionGroup as described in manual. tor stops with error.
torrc:
SocksPort 9051 SessionGroup=1
Jun 15 16:46:24.700 [warn] Invalid SocksPort option '"SessionGroup=INT"'
Jun 15 16:46:24.700 [warn] Failed to parse/validate config: Invalid SocksPort/SocksListenAddress configuration
Jun 15 16:46:24.701 [err] Reading config failed--see warnings above.
second try
SocksPort 9051 SessionGroup=INT
Results into:
Jun 15 16:46:35.677 [warn] Invalid SocksPort option '"SessionGroup=1"'
Jun 15 16:46:35.678 [warn] Failed to parse/validate config: Invalid SocksPort/SocksListenAddress configuration
Jun 15 16:46:35.677 [warn] Invalid SocksPort option '"SessionGroup=1"'
Jun 15 16:46:35.678 [warn] Failed to parse/validate config: Invalid SocksPort/SocksListenAddress configuration
Jun 15 16:46:35.678 [err] Reading config failed--see warnings above.
**Trac**:
**Username**: acceleraTorTor: 0.3.5.x-finalNick MathewsonNick Mathewson