Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T10:52:54Zhttps://gitlab.torproject.org/legacy/trac/-/issues/31369HSv3 descriptor support in stem [decoding]2020-06-13T10:52:54ZGeorge KadianakisHSv3 descriptor support in stem [decoding]Hello,
it would be great if stem could support v3 onion service descriptors: both to parse them and to generate them.
This would be particularly useful as part of onionbalance v3 (#26768). The old onionbalance actually generates the de...Hello,
it would be great if stem could support v3 onion service descriptors: both to parse them and to generate them.
This would be particularly useful as part of onionbalance v3 (#26768). The old onionbalance actually generates the descriptors itself in an ad-hoc way, but it would be great if we could have stem make them in the new one.
Damian asked me for an HSFETCH command that will fetch a stable v3 onion desc. I used Donncha's script from here (https://gist.github.com/DonnchaC/13b744a1e30b7d34bc26) like this:
`python tmp/hsfetch.py vww6ybal4bd7szmgncyruucpgfkqahzddi37ktceo3ah7ngmcopnpyyd.onion` and that gives me the riseup onion descriptor (there is no DDG v3 onion right now).
Also, v3 descriptors are more complicated than v2 descriptors because of their double-layer encryption. You can see more info here: https://github.com/torproject/torspec/blob/master/rend-spec-v3.txt#L1046
https://github.com/torproject/tor/blob/0acfd7dcee2a4473eba05a53d6df2d6d4fe2050b/src/feature/hs/hs_descriptor.c#L2639
Damian, let me know how that looks like to you and how we can help. We plan to start working on Onionbalance v3 at mid-to-late August, but if this is something we can have at a reasonaable timeframe we can potentially delay the descriptor parts of it for some time until stem support exists.Damian JohnsonDamian Johnsonhttps://gitlab.torproject.org/legacy/trac/-/issues/3733Tor should abandon rendezvous circuits that cause a client request to time out2020-06-13T14:12:26ZRobert RansomTor should abandon rendezvous circuits that cause a client request to time out```
Aug 14 05:59:04.639 [Notice] Rend stream is 120 seconds late. Giving up on address '[scrubbed].onion'.
Aug 14 05:59:09.631 [Notice] Rend stream is 120 seconds late. Giving up on address '[scrubbed].onion'.
Aug 14 06:04:08.637 [Notice...```
Aug 14 05:59:04.639 [Notice] Rend stream is 120 seconds late. Giving up on address '[scrubbed].onion'.
Aug 14 05:59:09.631 [Notice] Rend stream is 120 seconds late. Giving up on address '[scrubbed].onion'.
Aug 14 06:04:08.637 [Notice] Rend stream is 120 seconds late. Giving up on address '[scrubbed].onion'.
Aug 14 06:04:10.634 [Notice] Rend stream is 120 seconds late. Giving up on address '[scrubbed].onion'.
Aug 14 06:04:30.680 [Notice] Rend stream is 120 seconds late. Giving up on address '[scrubbed].onion'.
```
All of these streams had been attached to the same rendezvous circuit.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/24963dos: Block single hop clients at the intro point2020-06-13T15:20:54ZDavid Gouletdgoulet@torproject.orgdos: Block single hop clients at the intro pointThis is so we remove load from spammy single hop clients at the Intro point.
Opened after discussion in #24902.
Lets consider this also as a possible backport.This is so we remove load from spammy single hop clients at the Intro point.
Opened after discussion in #24902.
Lets consider this also as a possible backport.Tor: 0.4.2.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/25568hs: Lookup failure cache when introducing to an intro point2020-06-13T15:23:25ZDavid Gouletdgoulet@torproject.orghs: Lookup failure cache when introducing to an intro pointIt turns out that if a descriptor contains 10 times the same intro point, and that the first introduction attempt fails, we'll try to connect to the same failing intro point again for all subsequent remaining intro points.
The intro poi...It turns out that if a descriptor contains 10 times the same intro point, and that the first introduction attempt fails, we'll try to connect to the same failing intro point again for all subsequent remaining intro points.
The intro point failure cache was introduced to avoid such a situation but it is only used between two descriptors that is if an intro point failed from the first descriptor and that intro point is still present in the second descriptor fetched, we ignore it.
However, this situation is about the same intro point in the same descriptor. In normal circumstances, this can't happen but it is still allowed by the protocol.
One issue with this is that a malicious service would induce many circuits out of the client than necessary. This can be used, theoretically, for a client guard discovery attack.
This affects both v2 and v3.Tor: 0.4.3.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/26768Support onionbalance in HSv32020-06-13T15:27:55ZGeorge KadianakisSupport onionbalance in HSv3We are implementing onionbalance in v3! This is the master ticket.
[Description changed to not confuse people with the old design.]We are implementing onionbalance in v3! This is the master ticket.
[Description changed to not confuse people with the old design.]Tor: unspecifiedGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/legacy/trac/-/issues/26806Check if Tor clients sometimes send duplicate cells on rendezvous circuits: P...2020-06-13T15:28:06Zs7rCheck if Tor clients sometimes send duplicate cells on rendezvous circuits: Possible replay detected! An INTRODUCE2 cell with thesame ENCRYPTED section was seenAs my v3 onion service is getting more and more popular, I started to get:
```
[warn] Possible replay detected! An INTRODUCE2 cell with thesame ENCRYPTED section was seen 32 seconds ago. Dropping cell.
```
I am inclined to think that th...As my v3 onion service is getting more and more popular, I started to get:
```
[warn] Possible replay detected! An INTRODUCE2 cell with thesame ENCRYPTED section was seen 32 seconds ago. Dropping cell.
```
I am inclined to think that this is more like a bug in Tor, maybe due to a race condition, rather than a replay attack.
I also think this is what causes #15618 - dgoulet confirmed that the warning can be reproduced every time a second `ESTABLISH_RENDEZVOUS` is sent over the same circuit.
This can probably go away if we fix #21084. I am not sure if that should be a parent ticket here or not, please change if you feel like it. I think I still have yawning's tool and notes about how to reproduce #21084.Tor: 0.4.3.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/27251Add single-onion-v23-ipv6-md to make test-network-all2020-06-13T15:29:56ZteorAdd single-onion-v23-ipv6-md to make test-network-allWhen IPv6 single onion services work, we should add the single-onion-v23-ipv6-md to TEST_CHUTNEY_FLAVORS_IPV6in src/test/include.am.
We can also remove single-onion-ipv6-md, because it's redundant.When IPv6 single onion services work, we should add the single-onion-v23-ipv6-md to TEST_CHUTNEY_FLAVORS_IPV6in src/test/include.am.
We can also remove single-onion-ipv6-md, because it's redundant.Tor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/27680Explain how to use auth cookie for onion services2020-06-13T15:31:18ZtraumschuleExplain how to use auth cookie for onion serviceshttps://www.torproject.org/docs/tor-manual.html.en#HidServAuth
https://www.torproject.org/docs/tor-onion-service.html.en
#19757https://www.torproject.org/docs/tor-manual.html.en#HidServAuth
https://www.torproject.org/docs/tor-onion-service.html.en
#19757Tor: 0.4.1.x-finaltraumschuletraumschulehttps://gitlab.torproject.org/legacy/trac/-/issues/28841Write tool for onion service health assessment2020-06-13T15:35:37ZGeorge KadianakisWrite tool for onion service health assessmentWe've been getting lots of reports about bad reachability of onion services (e.g. #28730) and in particular the v3 ones.
We need a tool that we can use to evaluate and monitor the health of onion services. We should use it to verify how...We've been getting lots of reports about bad reachability of onion services (e.g. #28730) and in particular the v3 ones.
We need a tool that we can use to evaluate and monitor the health of onion services. We should use it to verify how reachable and stable onions are, and also as a benchmark for how their stability changes over time.
A relevant ticket here is #13209 which we can leverage in the future.
One way to write such a tool is to provide it with an onion service, and the tool fetches its desc from every HSDir, then introduces itself to all the intro points, and make sure that rendezvous can occur. Then it monitors this over time to find issues with reachability.Tor: unspecifiedDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/28966HSv3 client auth insufficiently documented (was: HiddenServiceAuthorizeClient...2020-06-13T15:36:05ZTracHSv3 client auth insufficiently documented (was: HiddenServiceAuthorizeClient incompatible)According to https://trac.torproject.org/projects/tor/ticket/20700#comment:17 this should be working.
HiddenServiceDir /var/lib/tor/keys/test
#HiddenServiceVersion 3
HiddenServicePort 80 127.0.0.1
HiddenServiceAuthorizeClient basic WUzZ...According to https://trac.torproject.org/projects/tor/ticket/20700#comment:17 this should be working.
HiddenServiceDir /var/lib/tor/keys/test
#HiddenServiceVersion 3
HiddenServicePort 80 127.0.0.1
HiddenServiceAuthorizeClient basic WUzZTg3OGQ
Dec 31 08:01:15.428 [notice] Tor 0.3.5.6-rc-dev (git-f4874765eabf1596) running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.1a, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.
Dec 31 08:01:15.428 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Dec 31 08:01:15.428 [notice] Read configuration file "/etc/tor/torrc".
Dec 31 08:01:15.431 [warn] Hidden service option HiddenServiceAuthorizeClient is incompatible with version 3 of service in /var/lib/tor/keys/test
Dec 31 08:01:15.431 [warn] Failed to parse/validate config: Failed to configure rendezvous options. See logs for details.
Dec 31 08:01:15.431 [err] Reading config failed--see warnings above.
**Trac**:
**Username**: rooTor: 0.4.2.x-finalGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/legacy/trac/-/issues/27471HS intermittently fails: Non-fatal assertion failed in send_introduce12020-06-13T15:36:10ZTracHS intermittently fails: Non-fatal assertion failed in send_introduce1When running 0.3.4.7-rc on Buster from deb.tpo
```
[WARN] Bug: /usr/bin/tor(_start+0x2a) [0x55edbc918eea]
[WARN] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7fd267a49b17]
[WARN] Bug: /usr/bin/tor(main+0x1...When running 0.3.4.7-rc on Buster from deb.tpo
```
[WARN] Bug: /usr/bin/tor(_start+0x2a) [0x55edbc918eea]
[WARN] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7) [0x7fd267a49b17]
[WARN] Bug: /usr/bin/tor(main+0x19) [0x55edbc918e99]
[WARN] Bug: /usr/bin/tor(tor_main+0x3a) [0x55edbc9190ea]
[WARN] Bug: /usr/bin/tor(tor_run_main+0x1005) [0x55edbc921155]
[WARN] Bug: /usr/bin/tor(do_main_loop+0x205) [0x55edbc91ea85]
[WARN] Bug: /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6(event_base_loop+0x5a7) [0x7fd26873f537]
[WARN] Bug: /usr/lib/x86_64-linux-gnu/libevent-2.1.so.6(+0x229ba) [0x7fd26873e9ba]
[WARN] Bug: /usr/bin/tor(+0x5271e) [0x55edbc91c71e]
[WARN] Bug: /usr/bin/tor(connection_handle_read+0xa73) [0x55edbc9decf3]
[WARN] Bug: /usr/bin/tor(+0x11e016) [0x55edbc9e8016]
[WARN] Bug: /usr/bin/tor(channel_tls_handle_cell+0x4db) [0x55edbc9a3e4b]
[WARN] Bug: /usr/bin/tor(command_process_cell+0x318) [0x55edbc9c2368]
[WARN] Bug: /usr/bin/tor(circuit_receive_relay_cell+0x2c4) [0x55edbc9448f4]
[WARN] Bug: /usr/bin/tor(+0x788bb) [0x55edbc9428bb]
[WARN] Bug: /usr/bin/tor(rend_process_relay_cell+0x293) [0x55edbc94b9b3]
[WARN] Bug: /usr/bin/tor(hs_client_receive_rendezvous_acked+0x79) [0x55edbca31a89]
[WARN] Bug: /usr/bin/tor(connection_ap_attach_pending+0x178) [0x55edbc9e1e58]
[WARN] Bug: /usr/bin/tor(connection_ap_handshake_attach_circuit+0x3bd) [0x55edbc9c0f3d]
[WARN] Bug: /usr/bin/tor(hs_client_send_introduce1+0x232) [0x55edbca31382]
[WARN] Bug: /usr/bin/tor(tor_bug_occurred_+0xb9) [0x55edbca6e499]
[WARN] Bug: /usr/bin/tor(log_backtrace+0x43) [0x55edbca537a3]
[WARN] Bug: Non-fatal assertion !(ip == NULL) failed in send_introduce1 at ../src/or/hs_client.c:559. Stack trace:
[WARN] tor_bug_occurred_(): Bug: ../src/or/hs_client.c:559: send_introduce1: Non-fatal assertion !(ip == NULL) failed.
```
**Trac**:
**Username**: tgragnatoTor: 0.4.2.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/29237Restore IPv6 intro points in the HS client tests2020-06-13T15:37:37ZteorRestore IPv6 intro points in the HS client testsIn #23576, I removed an IPv6-only intro point from the HS client tests. We should put it back in.In #23576, I removed an IPv6-only intro point from the HS client tests. We should put it back in.Tor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/29338restore HiddenServiceAuthorizeClient in v32020-06-13T15:37:55ZTracrestore HiddenServiceAuthorizeClient in v3According to the manual, for v3 hidden services, if the contents of <HiddenServiceDir>/authorized_clients/ cannot be loaded, then the Hidden Service is enabled and is accessible to anyone with the onion address. This is a security hole....According to the manual, for v3 hidden services, if the contents of <HiddenServiceDir>/authorized_clients/ cannot be loaded, then the Hidden Service is enabled and is accessible to anyone with the onion address. This is a security hole. It opens the possibility that the user intended for the service to require authorization, but due to files being moved or deleted or inaccessible or other file system problem, the hidden service incorrectly becomes accessible to anyone.
Please restore the configuration option HiddenServiceAuthorizeClient for v3 services. If it is set to "basic", then authentication should be required for the service regardless of whether <HiddenServiceDir>/authorized_clients/ can be read, or alternately, if the authorized users cannot be read, tor should not start up or should not enable the hidden service.
**Trac**:
**Username**: AlanTor: 0.4.2.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/29669hs: ADD_ONION with NEW:BEST is still pinned on v22020-06-13T15:39:01ZDavid Gouletdgoulet@torproject.orghs: ADD_ONION with NEW:BEST is still pinned on v2Even though the control spec says:
```
(The "NEW:BEST" option obeys the HiddenServiceVersion torrc option default
value. Since 0.3.5.1-alpha, it is 3. For Tor versions before 0.3.5.1-alpha,
default HiddenServiceVersion is 2.)
```
...Even though the control spec says:
```
(The "NEW:BEST" option obeys the HiddenServiceVersion torrc option default
value. Since 0.3.5.1-alpha, it is 3. For Tor versions before 0.3.5.1-alpha,
default HiddenServiceVersion is 2.)
```
... in control.c, the `ADD_ONION` command has this condition that basically pins the `NEW:BEST` to v2:
```
if (!strcasecmp(key_type_rsa1024, key_blob) ||
!strcasecmp(key_type_best, key_blob)) {
/* "RSA1024", RSA 1024 bit, also currently "BEST" by default. */
```
Not good! `NEW:BEST` should obey the default version, not something hardcoded like so. This will need a spec update to mention the correct tor version.Tor: 0.4.2.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/29998Objective 1, Activity 1.2: Adopt OnionBalance features into onion services v32020-06-13T15:40:10ZPili GuerraObjective 1, Activity 1.2: Adopt OnionBalance features into onion services v3This is the parent ticket to hold any tickets under this activity, including:
- Improving scalability of onion services by adopting OnionBalance features into onion services v3This is the parent ticket to hold any tickets under this activity, including:
- Improving scalability of onion services by adopting OnionBalance features into onion services v3Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29999Objective 1, Activity 2: Denial of service defences2020-06-13T15:40:11ZPili GuerraObjective 1, Activity 2: Denial of service defencesThis is the parent ticket to hold any tickets under this activity, including:
- Reducing the amount of circuits that they build over time on the Tor network
- Providing more ways for onion service administrators to control the influx of ...This is the parent ticket to hold any tickets under this activity, including:
- Reducing the amount of circuits that they build over time on the Tor network
- Providing more ways for onion service administrators to control the influx of incoming users in heavy traffic scenarios.
- Improving our defense mechanisms by:
- Decreasing onion service load on the Tor network, by slowing down Tor circuit creation on startup.
- Optimizing relevant onion service functions that are called multiple times therefore taking a lot of the CPU.
- Making it harder for adversaries to force services to rotate their introduction points.
- Writing a Tor software change proposal for a “rendezvous approver” API that can be useful for:
1. Rate limiting; allow at most N unauthenticated clients over a set time period
2. Extra-conservative logic like "stop accepting connections during potential guard discovery"
3. Limiting capacity to control server load; only allow N simultaneous clients.
4. Protocol-tuned rules for things like Ricochet
5. More advanced pre-rendezvous authorization
6. Load-balancing across multiple servers running Tor onion services
- Closing client circuit once the INTRO1/ACK dance has been completed, decreasing load on the Tor network.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/30281Sponsor27 master ticket2020-06-13T15:40:57ZGeorge KadianakisSponsor27 master ticketThis is the master ticket for the whole Sponsor27 project.This is the master ticket for the whole Sponsor27 project.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/30454hs-v3: INTRODUCE1 trunnel code doensn't handle HS_INTRO_ACK_STATUS_CANT_RELAY2020-06-13T15:41:25ZDavid Gouletdgoulet@torproject.orghs-v3: INTRODUCE1 trunnel code doensn't handle HS_INTRO_ACK_STATUS_CANT_RELAYI'm not sure how it happens because this code got in almost at the same time but the introduction ACK status code are not synchronized with what trunnel can parse which can lead to the intro point hard asserting() if it ever tries to sen...I'm not sure how it happens because this code got in almost at the same time but the introduction ACK status code are not synchronized with what trunnel can parse which can lead to the intro point hard asserting() if it ever tries to send the status code: `HS_INTRO_ACK_STATUS_CANT_RELAY = 0x0003`.
See `cell_introduce1.trunnel`, it does not handle status code `0x0003`:
```
u16 status IN [0x0000, 0x0001, 0x0002];
```
Fortunately for us, it can NOT happen with the current code. The only call site is here:
```
/* Relay the cell to the service on its intro circuit with an INTRODUCE2
* cell which is the same exact payload. */
if (relay_send_command_from_edge(CONTROL_CELL_ID, TO_CIRCUIT(service_circ),
RELAY_COMMAND_INTRODUCE2,
(char *) request, request_len, NULL)) {
log_warn(LD_PROTOCOL, "Unable to send INTRODUCE2 cell to the service.");
/* Inform the client that we can't relay the cell. */
status = HS_INTRO_ACK_STATUS_CANT_RELAY;
goto send_ack;
}
```
... and turns out that `relay_send_command_from_edge()` can *NEVER* return anything else than 0 (we've audited all the currently supported versions, they all behave the same there).
This prevents tor from asserting in `send_introduce_ack_cell()` with:
```
/* A wrong status is a very bad code flow error as this value is controlled
* by the code in this file and not an external input. This means we use a
* code that is not known by the trunnel ABI. */
tor_assert(ret == 0);
```
There are a series of things to fix here.
1. The status code should be defined within the trunnel file and ONLY that ABI should be used. In short, `hs_intro_ack_status_t` should disappear in favor of the one that needs to be defined with trunnel. Side node, same must be done for: `hs_intro_auth_key_type_t`
2. Because clients won't be able to know what `HS_INTRO_ACK_STATUS_CANT_RELAY` is, we should remove it at once for now since it was/is never used.
3. We should add a default case to the trunnel definition so if we ever extend this later, tor will not explode or simply fail to parse the cell.
4. We should conduct an audit of the `tor_assert()` in the HS code and replace them with non fatal ones if possible.
We got very lucky here because else it would have been an easy remote relay crash so (4) is very important. One lesson is also to *NEVER* duplicate any values defined in a trunnel file. Always use the defined ABI of that trunnel spec.
End note, this was found by #15516 branch which re-used this error code and during testing, the intro point relay exploded.
All this got into 0.3.0.1-alpha.Tor: 0.3.5.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/29034circuit: Cleanup an HS circuit when it is being re-purposed2020-06-13T15:42:10ZDavid Gouletdgoulet@torproject.orgcircuit: Cleanup an HS circuit when it is being re-purposedMike found out that when an IP/RP circuit fails to build in the right amount of time (for instance through `circuit_expire_building()`), it is re-purposed to become a measurement circuit.
The issue is that those HS circuits are set in t...Mike found out that when an IP/RP circuit fails to build in the right amount of time (for instance through `circuit_expire_building()`), it is re-purposed to become a measurement circuit.
The issue is that those HS circuits are set in the HS circuitmap and have an `hs_ident` or `rend_data` set to them that should really not linger in the circuit object if the circuit is not an HS one anymore.
Offenders: `circuit_build_times_mark_circ_as_measurement_only()` and `pathbias_send_usable_probe()`.
Solution:
`circuit_change_purpose()` is probably the right place to make a callback within the HS subsystem specific to cleaning up a circuit for a purpose change. I think we need a new function that specifically does that and not use `hs_circ_cleanup()` since it won't remove the ident.
Lingering circuits in the HS circuitmap is bad and this bug could probably explain some of the issues we had with clients unable to establish connections because the IP auth key wouldn't match the one in the circuit ident.
I strongly believe this should be backported up to 0.3.5 at the very least.Tor: 0.3.5.x-finalMike PerryMike Perryhttps://gitlab.torproject.org/legacy/trac/-/issues/30790hs-v3: Write a proposal for an ESTABLISH_INTRO cell extension containing DoS ...2020-06-13T15:42:15ZDavid Gouletdgoulet@torproject.orghs-v3: Write a proposal for an ESTABLISH_INTRO cell extension containing DoS defense parametersThis ticket is for writing the proposal that will allow an onion service to put DoS defense parameters (at the intro point, like #15516) in the `ESTABLISH_INTRO` cell so the intro point can use those.
It is more than possible that we pr...This ticket is for writing the proposal that will allow an onion service to put DoS defense parameters (at the intro point, like #15516) in the `ESTABLISH_INTRO` cell so the intro point can use those.
It is more than possible that we probably want more than just DoS parameters in that cell.
This is related to #15516 as the parameters, initially, come from that ticket work.Tor: 0.4.2.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.org