Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:21:57Zhttps://gitlab.torproject.org/legacy/trac/-/issues/25227Avoid storing all Tor nodes in RAM2020-06-13T15:21:57ZteorAvoid storing all Tor nodes in RAM```
<teor4> ahf: I've been thinking about reducing Tor's RAM usage by sampling from each consensus, rather than keeping it all in RAM
<teor4> there are ways to stream the consensus, pick a weighted sample, and just keep those nodes
<teor...```
<teor4> ahf: I've been thinking about reducing Tor's RAM usage by sampling from each consensus, rather than keeping it all in RAM
<teor4> there are ways to stream the consensus, pick a weighted sample, and just keep those nodes
<teor4> https://en.wikipedia.org/wiki/Reservoir_sampling
<teor4> it would really help the iOS VPN, and other embedded impls
<teor4> The catch is that if you run out of sampled nodes, you need to re-parse the consensus
<+ahf> teor4: i had been thinking a bit in the back of my head if we could refactor the code that is used for accessing the consensus to be a bit more pluggable to experiment with different ways of backing them
<+ahf> teor4: on ios i think we can mmap() files without them adding to the memory limit of the application too
<teor4> that would be helpful, because we could only keep the selected node queues in RAM
<teor4> if we got the queue size right, we could regenerate them once per consensus
<+ahf> do you want to create a ticket with this? maybe add sponsor8-can to the keywords
```
We could be smart about node selection, and just store some weights for each node. And then when we use too much RAM for nodes, we could evict older nodes.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/25153Specify how PrivCount in Tor statistics are configured and interpreted2020-06-13T15:21:41ZteorSpecify how PrivCount in Tor statistics are configured and interpretedIn ~~prop#280~~ prop#288, we specified the counters, noise, and secure aggregation for PrivCount in Tor.
Now we need to specify:
* how long each collection round goes for
* how we determine which counters are collected
* how we configur...In ~~prop#280~~ prop#288, we specified the counters, noise, and secure aggregation for PrivCount in Tor.
Now we need to specify:
* how long each collection round goes for
* how we determine which counters are collected
* how we configure the amount of noise for each counter
* how Metrics can interpret the final results
We should also try to answer the unanswered questions in prop280.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/23858Create a local tool that provides detailed statistics for relay operators2020-06-13T15:15:51ZteorCreate a local tool that provides detailed statistics for relay operatorsWe want to reduce relay stats to 24 hours, and maybe eventually a month.
Maybe we could use nyx or local Atlas-like graphs for this.We want to reduce relay stats to 24 hours, and maybe eventually a month.
Maybe we could use nyx or local Atlas-like graphs for this.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22798Windows relay is several times slower than Linux relay2020-06-13T15:13:25ZTracWindows relay is several times slower than Linux relayI have launched two relays: first one in native mode on Windows, second one in virtual machine on Linux.
Then measured their bandwidth using three-hop circuit: refEntry, myRelay, refExit
refEntry is [[https://atlas.torproject.org/#detai...I have launched two relays: first one in native mode on Windows, second one in virtual machine on Linux.
Then measured their bandwidth using three-hop circuit: refEntry, myRelay, refExit
refEntry is [[https://atlas.torproject.org/#details/13B2354C74CCE29815B4E1F692F2F0E86C7F13DD|13B2354C74CCE29815B4E1F692F2F0E86C7F13DD]]
refExit is [[https://atlas.torproject.org/#details/07C05ED4825F51D5BE4CDBBAA80BFA484132A2F5|07C05ED4825F51D5BE4CDBBAA80BFA484132A2F5]]
Windows version of Tor was able to provide 51 KiB/s.
Linux version - 163 KiB/s, which is three times higher.
But this was my measurements.
BwAuth ratings for this relays are far more different:
Windows one have weight = 18 (19/13/22/18).
Linux one got weight = 1030 (293/1030/1460).
Which leads to actual traffic rising from 1 KiB/s to ~500 KiB/s.
I can keep relay in virtual machine for a while, but it would be much better if Windows version gets fixed.
Here are the versions of software used in tests:
OS: Windows 7 SP1 x64 (host)
OS: Ubuntu 16.04 x64 (guest)
VM: VirtualBox 5.1.22
Tor: 0.2.9.11 (Linux)
Tor: 0.2.9.11, 0.3.0.8 (Windows)
Also I have obtained TCP packets dump from relay's network interface:
(REDACTED)
Packets 1-1584 are slow transfer (Windows relay).
Packets 1585-8659 are fast transfer (Linux VM relay).
I can made additional tests and provide additional information if needed.
**Trac**:
**Username**: VortTor: 0.2.9.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/22498Offline directory authorities need a way to post their certificate to other a...2020-06-13T15:10:01ZteorOffline directory authorities need a way to post their certificate to other authoritiesWe have wanted to be able to run (the signing parts of) a directory authority offline for a while, because it's more secure.
So I have been experimenting with an offline (ORPort and DirPort unreachable) directory authority on the test n...We have wanted to be able to run (the signing parts of) a directory authority offline for a while, because it's more secure.
So I have been experimenting with an offline (ORPort and DirPort unreachable) directory authority on the test net.
Almost everything works: it posts votes, downloads votes from other authorities, signs consensuses, and posts its signature. It could easily do these things using a 3-hop Tor path.
But once its authority certificate expires, it has no way to post it to the other authorities.
A workaround is to overwrite another authority's cached-certs file with the missing authority certificate file. But this is nasty.
We should make authorities accept certificate posts, and post their certificates to one another.Tor: very long termhttps://gitlab.torproject.org/legacy/trac/-/issues/22408Refactor functions over 300 lines long.2020-06-13T15:09:35ZNick MathewsonRefactor functions over 300 lines long.I think it's reasonable to impose a much smaller limit, but let's start by attacking the worst offenders. cc'ing catalyst because we've talked about this before.
It's probably a good idea to use a separate ticket or separate branch for...I think it's reasonable to impose a much smaller limit, but let's start by attacking the worst offenders. cc'ing catalyst because we've talked about this before.
It's probably a good idea to use a separate ticket or separate branch for each one.
```
300 connection_listener_new
306 networkstatus_set_current_consensus
327 rend_service_receive_introduction
330 ed_key_init_from_file
332 circuit_get_open_circ_or_launch
355 tor_spawn_background
360 router_dump_router_to_string
389 networkstatus_verify_bw_weights
389 parse_socks
399 connection_edge_process_relay_cell
404 circuit_expire_building
449 parse_port_config
535 options_act
541 connection_ap_handshake_rewrite_and_attach
548 router_parse_entry_from_string
638 networkstatus_parse_vote_from_string
973 networkstatus_compute_consensus
1269 options_validate
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/22407Support HTTP CONNECT tunnels as an alternative to SOCKS2020-06-13T15:09:35ZNick MathewsonSupport HTTP CONNECT tunnels as an alternative to SOCKS_Note:_ This is NOT a ticket about adding a caching or rewriting or privacy-enhancing HTTP proxy to Tor.
We may want to support the "HTTP CONNECT" command as an alternative to SOCKS, since a fair number of applications support the form..._Note:_ This is NOT a ticket about adding a caching or rewriting or privacy-enhancing HTTP proxy to Tor.
We may want to support the "HTTP CONNECT" command as an alternative to SOCKS, since a fair number of applications support the former but not the latter. For the current definition of HTTP CONNECT, see [RFC 7231](https://tools.ietf.org/html/rfc7231#section-4.3.6).
Our existing HTTP parsing code should be sufficient to handle these requests, though we would want to make sure we got the semantics right.
The flexibility of HTTP could make this an alternative to proposal 229.Tor: 0.3.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21358Tor fails to reconnect after computer resumes from sleep2020-06-13T15:05:51Zweasel (Peter Palfrader)Tor fails to reconnect after computer resumes from sleepA user reports that tor 0.2.9.x does not work after resumes, similar (or the same as) bug #19969, marked already fixed.
https://bugs.debian.org/853146:
> Tor (or tor with little t) doesn't reconnect automatically to Tor
> network after...A user reports that tor 0.2.9.x does not work after resumes, similar (or the same as) bug #19969, marked already fixed.
https://bugs.debian.org/853146:
> Tor (or tor with little t) doesn't reconnect automatically to Tor
> network after I resume my computer from sleep. So instead I have to
> run "sudo systemctl restart tor" to get it working again. Though, I
> think it reconnects automatically after several minutes of computer
> being on but I would like it to reconnect immediately after the
> computer has resumed from sleep.
>
> I'm not sure if tor uses systemd but so if it uses maybe there is some
> 'resume sleep' hook in systemd that could be used for tor to
> reconnect. Please let me know if you think such change would cause
> some other problems.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/21179Add a fuzzing harness for the tor OR protocol2020-06-13T15:05:18ZteorAdd a fuzzing harness for the tor OR protocolTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/19881New guard plan - guard selection for circuits2020-06-13T15:00:05ZAndrea ShepardNew guard plan - guard selection for circuitsNew guard plan summarized at https://lists.torproject.org/pipermail/tor-dev/2016-July/011234.html
5) Selecting guards for circuits
- Meant to replace choose_random_entry_impl()
- See section SELECTING in prop271
- Add new circuit sta...New guard plan summarized at https://lists.torproject.org/pipermail/tor-dev/2016-July/011234.html
5) Selecting guards for circuits
- Meant to replace choose_random_entry_impl()
- See section SELECTING in prop271
- Add new circuit states to or_circuit_t
- Implement the guard selection logic
- Unittests on circuit state machine
- Unittests on guard selection logicTor: 0.3.0.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/17945Stop single hop client connections to Single Onion Services2020-06-13T14:59:25ZteorStop single hop client connections to Single Onion ServicesTor2Web clients make a one-hop connection to HSDirs, intro points, and rend points. Single Onion Services also make a one-hop connection to the rendezvous point.
This uses Tor as a one-hop proxy (in this case, to a single onion service)...Tor2Web clients make a one-hop connection to HSDirs, intro points, and rend points. Single Onion Services also make a one-hop connection to the rendezvous point.
This uses Tor as a one-hop proxy (in this case, to a single onion service), which we try to avoid, because it enables certain attacks. We also try to avoid single hop connections in the onion service protocol, because they give IP addresses to middle relays.
See the child tickets for details.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/19377Consider retry/backoff behavior when building new circuits2020-06-13T14:58:36ZAndrea ShepardConsider retry/backoff behavior when building new circuitsRetrying connections is the wrong level of abstraction at which to think about circuit failure behavior IMO (see comment on #15942); we should consider whether, as a client or an HS, we're ever doing anything like repeatedly retrying to ...Retrying connections is the wrong level of abstraction at which to think about circuit failure behavior IMO (see comment on #15942); we should consider whether, as a client or an HS, we're ever doing anything like repeatedly retrying to build a circuit without smart backoff behavior for the sake of DoS resistance though.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/19327controller: expose fine-grained circuit detail.2020-06-13T14:58:32ZNick Mathewsoncontroller: expose fine-grained circuit detail.circuits have lots of fields on them, and not all are currently exposed via getinfo. For testing, it might be useful to list more.circuits have lots of fields on them, and not all are currently exposed via getinfo. For testing, it might be useful to list more.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/19286Document circuit abstraction, completely2020-06-13T14:58:13ZNick MathewsonDocument circuit abstraction, completelyWe should document how exactly circuits work as a part of our overall documentation efforts.We should document how exactly circuits work as a part of our overall documentation efforts.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/17799Use a better PRNG unless OpenSSL starts using a better one on their own.2020-06-13T14:52:10ZteorUse a better PRNG unless OpenSSL starts using a better one on their own.#17694 hashes important PRNG output with some system randomness before use, so that observed PRNG outputs are resistant to PRNG state analysis.
But almost all of Tor's use of PRNG outputs is observable from one or more locations outside...#17694 hashes important PRNG output with some system randomness before use, so that observed PRNG outputs are resistant to PRNG state analysis.
But almost all of Tor's use of PRNG outputs is observable from one or more locations outside Tor, whether in salts or nonces sent to other machines on the wire, or in the random choices made in guard, directory, and path selection.
We could hash all of the bytes coming from the PRNG to avoid this state exposure. (Although we might not need to use the system randomness source each time.)Tor: unspecifiedNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/6676Nuke ‘MyFamily’2020-06-13T14:43:50ZRobert RansomNuke ‘MyFamily’For reasons discussed in #5565, the ‘MyFamily’ option should go away:
* It is probably harmful for clients to refuse to use two relays operated by the same honest entity, and malicious relay operators will not use MyFamily honestly.
*...For reasons discussed in #5565, the ‘MyFamily’ option should go away:
* It is probably harmful for clients to refuse to use two relays operated by the same honest entity, and malicious relay operators will not use MyFamily honestly.
* Honest relay operators have a hard time setting MyFamily correctly on all of their relays.
* Relay family information bloats up microdescriptors, likely for no benefit.
* Groups of relays operated by the same entity can be recognized tolerably well (e.g. for metrics purposes) using their ContactInfo string instead (modulo the possible need for fuzzy matching of some sort).
This change requires (assuming I didn't forget anything):
* a proposal, spec change, and consensus method to remove relay family information from microdescriptors;
* a corresponding code change for Tor dirauths;
* a code change to clients to make them ignore relay family information (if they receive it, e.g. by turning off UseMicrodescriptors) by default, and warn if the user tries to re-enable use of relay family information without disabling UseMicrodescriptors.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/5565MyFamily should provide an alternate non-idhex subscription mechanism2020-06-13T14:43:50ZMike PerryMyFamily should provide an alternate non-idhex subscription mechanismEverybody hates MyFamily. It's cumbersome, hard to update, hard to spot check that it's correct, and it gets in the way of vastly improving the practical security of the network through ephemeral identity keys (#5563).
So first off, wha...Everybody hates MyFamily. It's cumbersome, hard to update, hard to spot check that it's correct, and it gets in the way of vastly improving the practical security of the network through ephemeral identity keys (#5563).
So first off, what is wrong with making this PoS an arbitrary token ("OurFamily" anyone?) If weirdos start joining families that people don't want them to, can't we just de-list those nodes?
If we really can't handle the risk of people joining arbitrary families for any period of time, we could deploy a signature scheme where a node has to sign its IP+OrPort, current idhex, and/or nickname using a family key and place that signature into its MyFamily field.
We could even make this an incrementally deployable solution. We could first make the new field free-form, and then later update it to require authentication with a family key.
But my guess is this is not worth significant engineering, and we should just make it a free-form token and de-list nodes who try to adopt themselves into random families without consent.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/13705Allow relays to promise in their descriptor that their IP address won't change2020-06-13T14:40:05ZRoger DingledineAllow relays to promise in their descriptor that their IP address won't changeImagine the following scenario: Oscar runs a fast relay that gets the Guard flag and accumulates some users, including a user Alice. Then some attacker does a guard enumeration attack to identify that his victim is using Oscar's relay as...Imagine the following scenario: Oscar runs a fast relay that gets the Guard flag and accumulates some users, including a user Alice. Then some attacker does a guard enumeration attack to identify that his victim is using Oscar's relay as her guard. He can get a warrant to collect Oscar's computer, but for whatever reason he's not allowed to tap the relay in-place. So he steals the computer, takes it to his location, turns it back on, and the relay starts up again. Alice then says "oh good, my guard is back online" and moves back to using it.
One straightforward option to reduce the risk of this scenario happening in practice is for relays that intend to have a static IP address to set a line in their descriptor that tells the directory authorities to refuse them if they show up from a different IP address. The implementation on the directory authority side would be to add the IP address to fingerprint mapping to the router-stability file or equivalent, and then check whether there's a mapping when considering newly published descriptors.
This idea wouldn't handle the attack when done on relays with dynamic or varying IP addresses.
Another avenue for addressing the attack is the encrypted identity key proposal and friends. I'm not sure if they handle this issue, or are orthogonal, or would supersede this idea.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/7971review address lists in tor_addr_is_internal_()2020-06-13T14:37:05Zcypherpunksreview address lists in tor_addr_is_internal_()Tor's common/address.c's tor_addr_is_internal_() might be a bit dated, regarding it's list of IANA special-purpose registry, and the IETF RFCs/I-Ds it uses. That code looks for reserved/localhost addresses, and mentions RFCs: 1918, 3879...Tor's common/address.c's tor_addr_is_internal_() might be a bit dated, regarding it's list of IANA special-purpose registry, and the IETF RFCs/I-Ds it uses. That code looks for reserved/localhost addresses, and mentions RFCs: 1918, 3879, 4193, and 4291, all of which are outdated.
Yesterday's I-D draft-bonica-special-purpose-06 <http://tools.ietf.org/id/draft-bonica-special-purpose-06.txt> is a list of all of the addresses, which are in RFC5735 (IPv4 addresses) and RFC5156 (IPv6 addresses). The I-D lists 16 addresses for IPv4 and 12 addresses for IPv6.
Tor appears to handle 7 IPv4 addresses (not 16), and 5 IPv6 addresses (not 12); and I don't think one of those (FEC0/10) is shared between the Tor and I-D lists, and might be either a Tor bug or an IETF I-D bug, or my misreading).
Someone who has a better understanding of how Tor uses local addresses, might want to review Tor's code, alongside a current I-D, to see if any of those missing addresses should be added. The I-D has more data than the below tables, so more helpful for deciding.
Tor IPv4 cases:
"0.0.0.0"
"10/8"
"0/8"
"127/8"
"169.254/16"
"172.16/12"
"192.168/16"
I-D IPv4 cases:
"0.0.0.0/8" (RFC 1122: 'This' Network)
"10.0.0.0/8" (RFC 1918: Private-Use)
"100.64.0.0/10" (RFC 6598: Shared Address Space)
"127.0.0.0/8" (RFC 1122: Loopback)
"169.254.0.0/16" (RFC 3927: Link Local)
"172.16.0.0/12" (RFC 1122: Private-Use)
"192.0.0.0/24" (RFC 5736: IETF Protocol Assignments)
"192.0.0.0/29" (RFC 6333: DS-Lite)
"192.0.2.0/24" (RFC 5737: Documentation (TEST-NET-1))
"192.88.99.0/24" (RFC 3068: 6to4 Relay Anycast)
"192.168.0.0/16" (RFC 1918: Private-Use)
"198.18.0.0/15" (RFC 2544: Benchmarking)
"198.51.100.0/24" (RFC 5737: Documentation (TEST-NET-2))
"203.0.113.0/24" (RFC 5737: Documentation (TEST-NET-3))
"240.0.0.0/4" (RFC 1112: Reserved)
"255.255.255.255/32" (RFC 0919: Limited Broadcast)
Tor IPv6 cases:
"::"
"::/127"
"fc00/7"
"fe80/10"
"fec0/10"
I-D IPv6 cases:
"::1/128" (RFC 4291: Loopback Address)
"::/128" (RFC 4291: Unspecified Address)
"::FFFF:0:0/96" (RFC 4291: IPv4-mapped Address)
"0100::/64" (RFC 6666: Discard-Only Prefix)
"2001:0000::/23" (RFC 2928: IETF Protocol Assignments)
"2001:0000::/32" (RFC 4380: TEREDO)
"2001:0002::/48" (RFC 5180: Benchmarking)
"2001:db8::/32" (RFC 3849: Documentation)
"2001:10::/28" (RFC 4843: ORCHID)
"2002::/16" (RFC 3056: 6to4)
"FC00::/7" (RFC 4193: Unique-Local)
"FE80::/10" (RFC 4291: Linked-Scoped Unicast)
Thanks,
LeeTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/10186Backtrace support for windows2020-06-13T14:33:00ZNick MathewsonBacktrace support for windowsWith #9299, we added backtrace support for nearly all operating systems. Naturally, this doesn't include windows, because Windows Is Different.
We should use StackWalk64 to walk the stack, and we should use SetUnhandledException filtuer...With #9299, we added backtrace support for nearly all operating systems. Naturally, this doesn't include windows, because Windows Is Different.
We should use StackWalk64 to walk the stack, and we should use SetUnhandledException filtuer to detect crashes, or whatever the preferred mechanisms are.Tor: unspecifiedTom Rittertom@ritter.vgTom Rittertom@ritter.vg