Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T14:02:49Zhttps://gitlab.torproject.org/legacy/trac/-/issues/1121reason unexpected while uploading descriptor2020-06-13T14:02:49ZTracreason unexpected while uploading descriptorHi,
since i upgraded my relay to 0.2.2.5-alpha i am getting these warn message below.
Oct 12 13:15:10.985 [notice] Bootstrapped 90%: Establishing a Tor circuit.
Oct 12 13:15:18.207 [notice] Tor has successfully opened a circuit. Looks...Hi,
since i upgraded my relay to 0.2.2.5-alpha i am getting these warn message below.
Oct 12 13:15:10.985 [notice] Bootstrapped 90%: Establishing a Tor circuit.
Oct 12 13:15:18.207 [notice] Tor has successfully opened a circuit. Looks like client functionality is working.
Oct 12 13:15:18.207 [notice] Bootstrapped 100%: Done.
Oct 12 13:16:09.666 [warn] http status 404 ("Not Found") reason unexpected while uploading descriptor to server '194.109.206.212:80').
Oct 12 13:16:17.277 [notice] Self-testing indicates your DirPort is reachable from the outside. Excellent.
Oct 12 13:16:18.508 [notice] Performing bandwidth self-test...done.
Oct 12 13:17:10.924 [warn] http status 404 ("Not Found") reason unexpected while uploading descriptor to server '194.109.206.212:80').
Oct 12 13:23:16.169 [warn] http status 404 ("Not Found") reason unexpected while uploading descriptor to server '194.109.206.212:80').
Some kind of new bug?
[Automatically added by flyspray2trac: Operating System: Other Linux]
**Trac**:
**Username**: dragonflyTor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/1922torrc.d-style configuration directories2020-06-13T16:52:58ZTractorrc.d-style configuration directoriestorrc.d-style configuration directories
**Trac**:
**Username**: aa138346torrc.d-style configuration directories
**Trac**:
**Username**: aa138346Tor: 0.3.1.x-finalJigsaw52Jigsaw52https://gitlab.torproject.org/legacy/trac/-/issues/2045Make tor_check_port_forwarding handle incomplete lines2020-06-13T15:09:26ZSteven MurdochMake tor_check_port_forwarding handle incomplete linesThe code for parsing the output of tor-fw-helper uses fgets() on a non-blocking file descriptor to fetch each line of output. This appears to not handle incomplete lines properly. Rather than buffering data until a complete line is retur...The code for parsing the output of tor-fw-helper uses fgets() on a non-blocking file descriptor to fetch each line of output. This appears to not handle incomplete lines properly. Rather than buffering data until a complete line is returned, fgets will return an incomplete line and set errno to be EAGAIN. fgets() should be replaced with something which can handle incomplete lines.
Issue was originally raised by nickm in bug #1903 and there is some further discussion in that ticket.Tor: 0.3.1.x-finalSteven MurdochSteven Murdochhttps://gitlab.torproject.org/legacy/trac/-/issues/4998MyFamily as a list2020-06-13T14:17:04Zweasel (Peter Palfrader)MyFamily as a listHi,
MyFamily currently is a config string. It'd be nice if it was a line list like RecommendedVersions as that would make configuration files a bit more readable.Hi,
MyFamily currently is a config string. It'd be nice if it was a line list like RecommendedVersions as that would make configuration files a bit more readable.Tor: 0.3.1.x-finalJigsaw52Jigsaw52https://gitlab.torproject.org/legacy/trac/-/issues/6298Tests should not depend on localhost==127.0.0.1 (Test 'addr/basic' failure in...2020-06-13T14:20:52ZAndrea ShepardTests should not depend on localhost==127.0.0.1 (Test 'addr/basic' failure in tor master)I just built branch 'master' (revision 46434ecf5b6f1ad88deb86fdac044c41ae4b534b) from the tor.git repository with gcc 4.5.3 on x86_64-linux using this configure line:
CC="gcc -m64 -mtune=core2" PKG_CONFIG_LIBDIR=/usr/lib64/pkgconfig PKG...I just built branch 'master' (revision 46434ecf5b6f1ad88deb86fdac044c41ae4b534b) from the tor.git repository with gcc 4.5.3 on x86_64-linux using this configure line:
CC="gcc -m64 -mtune=core2" PKG_CONFIG_LIBDIR=/usr/lib64/pkgconfig PKG_CONFIG_PATH=/usr/share/pkgconfig ./configure --prefix=/usr --libdir=/usr/lib64 --libexecdir=/usr/libexec64 --sysconfdir=/etc --localstatedir=/var --build=x86_64-linux --host=x86_64-linux --disable-linker-hardening --disable-asciidoc
This causes a test suite failure I haven't seen before:
addr/basic:
FAIL test_addr.c:36: assert((u32) == (0x7f000001u)): 2851995905 vs 2130706433
[basic FAILED]
Turns out the DNS in this hotel *resolves localhost to 169.254.1.1*. This is horribly wrong, of course, and I should have had a proper /etc/hosts for it anyway (*lashes self with wet noodle*), but why does the test suite depend on stuff random machines elsewhere on the network do at all to pass? Shouldn't we mock this stuff somehow?Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/6892improve torify manpage to say different connections may be over the same circ...2020-06-13T14:22:59Zweasel (Peter Palfrader)improve torify manpage to say different connections may be over the same circuit.Linus Lüssing would like the torify manpage to spell out that it does not do anything to make different torify calls unlinkable.
Probably not an unreasonable request.
For details of his request see
http://bugs.debian.org/681123
Cheers...Linus Lüssing would like the torify manpage to spell out that it does not do anything to make different torify calls unlinkable.
Probably not an unreasonable request.
For details of his request see
http://bugs.debian.org/681123
Cheers,
weaselTor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/8240Raise our guard rotation period, if appropriate2022-03-22T13:02:42ZRoger DingledineRaise our guard rotation period, if appropriateTariq's COGS paper from WPES 2012 shows that a significant component of guard churn is due to voluntary rotation, rather than actual network changes:
http://freehaven.net/anonbib/#wpes12-cogs
In short, if the target client makes sensiti...Tariq's COGS paper from WPES 2012 shows that a significant component of guard churn is due to voluntary rotation, rather than actual network changes:
http://freehaven.net/anonbib/#wpes12-cogs
In short, if the target client makes sensitive connections continuously every day for months, and you (the attacker) run some fast guards, the odds get pretty good that you'll become the client's guard at some point and get to do a correlation attack.
We could argue that the "continuously every day for months" assumption is unrealistic, so in practice we don't know how bad this issue really is. But for hidden services, it could well be a realistic assumption.
There are going to be (at least) two problems with raising the guard rotation period. The first is that we unbalance the network further wrt old guards vs new guards, and I'm not sure by how much, so I'm not sure how much our bwauth measurers will have to compensate. The second (related) problem is that we'll expand the period during which new guards don't get as much load as they will eventually get. This issue already results in confused relay operators trying to shed their Guard flag so they can resume having load.
In sum, if we raise the rotation period enough that it really results in load changes, then we could have unexpected side effects like having the bwauths raise the weights of new (and thus totally unloaded) guards to huge numbers, thus ensuring that anybody who rotates a guard will basically for sure get one of these new ones.
The real plan here needs a proposal, and should be for 0.2.5 or later. I wonder if we can raise it 'some but not too much' in the 0.2.4 timeframe though?Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/12931TOR_PT_SERVER_TRANSPORT_OPTIONS are not escaped according to pt-spec.txt2020-06-13T14:38:03ZYawning AngelTOR_PT_SERVER_TRANSPORT_OPTIONS are not escaped according to pt-spec.txtpt-spec.txt:
```
"TOR_PT_SERVER_TRANSPORT_OPTIONS" -- A semicolon-separated list
of <key>:<value> pairs, where <key> is a transport name and
<value> is a k=v string value with options that are to be passed
to t...pt-spec.txt:
```
"TOR_PT_SERVER_TRANSPORT_OPTIONS" -- A semicolon-separated list
of <key>:<value> pairs, where <key> is a transport name and
<value> is a k=v string value with options that are to be passed
to the transport. Colons, semicolons, equal signs and backslashes
MUST be escaped with a backslash. TOR_PT_SERVER_TRANSPORT_OPTIONS
is optional and might not be present in the environment of the
proxy if no options are need to be passed to transports.
```
or/transports.c:get_transport_options_for_server_proxy()
```
char *escaped_opts = tor_escape_str_for_pt_args(options, ":;\\");
```
Equal signs are not escaped. I'm not sure if anything will break with different behavior.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/12970dir-spec's a-line shouldn't mention portlists2020-06-13T14:38:07ZDamian Johnsondir-spec's a-line shouldn't mention portlistsReally quick thing [spotted by joelanders](https://trac.torproject.org/projects/tor/ticket/9380#comment:24). The dir-spec's [description of an a-line](https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt#l1591) says...
```
...Really quick thing [spotted by joelanders](https://trac.torproject.org/projects/tor/ticket/9380#comment:24). The dir-spec's [description of an a-line](https://gitweb.torproject.org/torspec.git/blob/HEAD:/dir-spec.txt#l1591) says...
```
Address and portlist are as for "or-address" as specified in
section 2.1.1.
```
Unless I'm missing something this (and or-address) only accept a single address:port, not a list of ports.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/13339Prop140: Complete Consensus diffs / Merge GSoC project2020-06-13T14:39:24ZmvdanProp140: Complete Consensus diffs / Merge GSoC projectGoogle Summer of Code finished over a month ago, and during this time I've been tidying up my code a bit and reading it for the merge. You will find it on github:
https://github.com/mvdan/tor
This ticket is for the sole purpose of foll...Google Summer of Code finished over a month ago, and during this time I've been tidying up my code a bit and reading it for the merge. You will find it on github:
https://github.com/mvdan/tor
This ticket is for the sole purpose of following the merge process and its progress. But as always I'm on IRC and mail if you want to contact me directly.
I just rebased against master this morning. Nick and Sebastian have been reviewing my code over the summer, but of course more sets of eyes are needed.
The test coverage for the diff generation and application is fine (see test_consdiff.c), but there aren't any tests for the stuff I wrote to wire it into serving and fetching consensus diffs. Not really sure how to go about that, can't really promise I'd have the time to dive into it.
And regarding commit messages and changelog entries, I pretty much went with my instinct. Chances are they can be improved - the commit messages for future reference and the changelog entries for future release changelogs - so criticism is welcome.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/13790Refactor and add comments to new_route_len()2020-06-13T14:40:22ZDavid Gouletdgoulet@torproject.orgRefactor and add comments to new_route_len()(function in src/or/circuitbuild.c)
At the moment, this function is kind of scary and it should be refactored and heavily documented to avoid future confusion. (This ticket exists because two different developers at different time span ...(function in src/or/circuitbuild.c)
At the moment, this function is kind of scary and it should be refactored and heavily documented to avoid future confusion. (This ticket exists because two different developers at different time span over a week expressed concerns about the behavior of HS and this function).
Bit of info on `new_route_len()`, this function is called to possibly extend a circuit with an extra hop that matches specific purpose(s). Right now it's testing the given purpose if it is *not* ESTABLISH_INTRO and TESTING, it will extend it to 4 hops (with exit information).
However, there are other purposes that do NOT need 4 hops. Fortunately, it seems that there are no code paths that end up calling this function with exit info and a purpose that should not be extended (investigated by special, sysrqb, asn and me). But, we all agree that this is very fragile thus the purpose condition in this function should be applied on only purposes that need 4 hops.
Furthermore, add more comments to make sure no more confusion happens with that fairly important function.Tor: 0.3.1.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/legacy/trac/-/issues/13802Add general trace-event logging instrumentation to Tor2020-06-13T14:40:27ZDavid Gouletdgoulet@torproject.orgAdd general trace-event logging instrumentation to TorRelevant to #13792, we (nickm and I) discussed what would be the way to collect and gather those statistics and measurements in a private network for performance analysis and debugging.
This would be completely deactivated at compile ti...Relevant to #13792, we (nickm and I) discussed what would be the way to collect and gather those statistics and measurements in a private network for performance analysis and debugging.
This would be completely deactivated at compile time meaning tracepoints would be NOP. You would have to explicitly enable that feature during the configure process.
We think the best way to go with that, in terms of code, is to go in a header with something that could look like this:
```
#ifdef TOR_USE_LTTNG
tracepoint_aname(arg1, arg2, [...]) tracepoint(aname, arg1, arg2, ...)
#else
tracepoint_aname(arg1, arg2, [...])
#endif
```
The really good thing about this approach is that 1) everything is centralized in one place, 2) you NOP the call if not configured thus no performance issue, 3) We can support lots of different tracers as well as shadow. It provides a simple way to hook any other tool in the trace event facility of the code.Tor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/13966Publish guidelines for reporting exploits2020-06-15T23:22:43ZTracPublish guidelines for reporting exploitsThere exists no easy to find documentation (on the wiki nor elsewhere) that advises how to report a suspected Tor (proxy, browser, bundle, transport...) exploit. And no search on keyservers shows up a 'security' key for a tor-sec@torproj...There exists no easy to find documentation (on the wiki nor elsewhere) that advises how to report a suspected Tor (proxy, browser, bundle, transport...) exploit. And no search on keyservers shows up a 'security' key for a tor-sec@torproject.org or similar account.
A blueprint for working this task could be: Just figure out how we've been handling exploit reporting in the past (tor-assistants@ maybe?) and make sure it's a consensus, and write it down in the wiki.
**Trac**:
**Username**: michaelTor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/15157Spec issues with consensus' new 'package' field2020-06-13T14:43:58ZDamian JohnsonSpec issues with consensus' new 'package' fieldHi Nick. Brought these up on irc but moving to a ticket. Our consensus' new unused [packages field](https://gitweb.torproject.org/torspec.git/commit/?id=ab6453476066fd1bf5c8cb577863c0cdd5079e0f) has some spec issues...
* It isn't very f...Hi Nick. Brought these up on irc but moving to a ticket. Our consensus' new unused [packages field](https://gitweb.torproject.org/torspec.git/commit/?id=ab6453476066fd1bf5c8cb577863c0cdd5079e0f) has some spec issues...
* It isn't very future proof. It's fine for now, but I'm not sure how it could validly have new attributes in the future.
* Presently it necessitates at least one DIGEST entry. Think that's probably a mistake.
* 'one or more non-=, non-" " characters' => That's too imprecise. For instance, it allows newlines which I'm sure wouldn't really be valid.
* It would be nice if it said if a DIGESTTYPE can appear multiple times. I'd like to model this as a hash but I'm not sure if DIGESTTYPE are unique.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/16082man page says "bandwidth accounting values" in state file are "unused so far"2020-06-13T14:46:20ZRoger Dingledineman page says "bandwidth accounting values" in state file are "unused so far"In the Tor man page, under DataDirectory/state, it says
```
· The current bandwidth accounting values (unused so far; see
below).
```
and then below is a DataDirectory/bw_accounting section, which says
```
...In the Tor man page, under DataDirectory/state, it says
```
· The current bandwidth accounting values (unused so far; see
below).
```
and then below is a DataDirectory/bw_accounting section, which says
```
Used to track bandwidth accounting values (when the current period
starts and ends; how much has been read and written so far this
period). This file is obsolete, and the data is now stored in the
'state' file as well. Only used when bandwidth accounting is
enabled.
```
So which is it?
I think it's in the state file, and we should stop saying that it's unused there.
I think.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/16706Too many connection_edge_process_relay_cell warnings2020-06-13T15:07:45Zs7rToo many connection_edge_process_relay_cell warningsHosting multiple hidden services on a Debian server running Tor 0.2.6.10 (no special setup or config options). I see thousands of such lines in the log files:
```
Jul 31 07:55:39.000 [warn] connection_edge_process_relay_cell (at origin) ...Hosting multiple hidden services on a Debian server running Tor 0.2.6.10 (no special setup or config options). I see thousands of such lines in the log files:
```
Jul 31 07:55:39.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 07:57:59.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:00:36.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:02:49.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:05:12.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:07:30.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:09:49.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:12:09.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:14:30.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:16:50.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:19:11.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:21:45.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:24:14.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:26:27.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:29:01.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:31:18.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:33:38.000 [warn] connection_edge_process_relay_cell (at origin) failed.
Jul 31 08:35:58.000 [warn] connection_edge_process_relay_cell (at origin) failed.
```
All hidden services are working and accessible, didn't reload/restart Tor. What is concerning is that there are so so many of these messages in a very short time. #9635 mentioned this as well, but along with other error messages which were indicating a wrong nTor key. Here we only have a single line, heavily repeated at short time intervals.Tor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/16858Non-ascii country code in extrainfo descriptor2020-06-13T15:30:35ZDamian JohnsonNon-ascii country code in extrainfo descriptorHi, starting recently (fifteen hours ago) torzurwelt started publishing extrainfo descriptors with a non-ascii country code in its dirreq-v3-reqs lines...
```
extra-info torzurwelt EA63329F9E4DC3C7366BC9244AA92B61F9BE77B1
published 2015...Hi, starting recently (fifteen hours ago) torzurwelt started publishing extrainfo descriptors with a non-ascii country code in its dirreq-v3-reqs lines...
```
extra-info torzurwelt EA63329F9E4DC3C7366BC9244AA92B61F9BE77B1
published 2015-08-19 05:50:58
...
geoip-db-digest C1EB5237F2FBAF63381D8551157F13D12EFCCA25
geoip6-db-digest 1F99B6B0EC78E9DB34D61AE7E0FC261D558E8E5D
dirreq-stats-end 2015-08-18 18:17:14 (86400 s)
dirreq-v3-ips de=16,it=16,us=16,??=8,ar=8,at=8,au=8,be=8,bg=8,br=8,ch=8,cz=8,es=8,fr=8,gb=8,ie=8,in=8,ir=8,jp=8,lt=8,ma=8,nl=8,pe=8,pl=8,pt=8,re=8,ro=8,ru=8,sa=8,tn=8,tt=8,tw=8,uy=8,ve=8
dirreq-v3-reqs ñÏ=32,de=16,it=16,us=16,??=8,ar=8,at=8,au=8,be=8,bg=8,br=8,ch=8,cz=8,es=8,fr=8,gb=8,ie=8,in=8,ir=8,jp=8,lt=8,ma=8,nl=8,pe=8,pl=8,pt=8,re=8,ro=8,ru=8,sa=8,tn=8,tt=8,tw=8,uy=8,ve=8
dirreq-v3-resp ok=104,not-enough-sigs=0,unavailable=0,not-found=0,not-modified=24,busy=8
dirreq-v3-direct-dl complete=0,timeout=0,running=0
```
Not sure how these are slipping in but pretty sure the authorities should reject these as malformed.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/16861Pad Tor connections to collapse netflow records2020-06-13T14:54:41ZMike PerryPad Tor connections to collapse netflow recordsThe collection of traffic statistics from routers is quite common. Recently, there was a minor scandal when a University network administrator upstream of UtahStateExits (and UtahStateMeekBridge) posted that they had collected over 360G ...The collection of traffic statistics from routers is quite common. Recently, there was a minor scandal when a University network administrator upstream of UtahStateExits (and UtahStateMeekBridge) posted that they had collected over 360G of netflow records to boingboing:
https://lists.torproject.org/pipermail/tor-relays/2015-August/007575.html
Unfortunately, the comment has since disappeared, but the tor-relays archives preserve it.
This interested me, so I asked some questions about the defaults and record resolution, and did some additional searching. It turns out that Cisco IOS routers have an "inactive flow timeout" that by default is 15 seconds, and it can't be set lower than 10 seconds. What this timeout does is cause the router to emit a new netflow "record" for a connection that is idle for that long, even if it stays open. Several other routers have similar settings. The Fortinet default is also 15 seconds for this. For Juniper, it is also 30 seconds (but Juniper routers can set it as low as 4 seconds).
With this information, I decided to write a patch that sends padding on a client's Tor connection bidirectionally at a random interval that we can control from the consensus, with a default of 4s-14s. It only sends padding if the connection is idle. It does not pad connections that are used only for tunneled directory traffic.
It also gives us the ability to control how long we keep said connections open. Since the default netflow settings for Cisco also generate a record for active flows after 30 minutes, it doesn't make a whole lot of sense to pad beyond that point.
This should mean that the total overhead for this defense is very low, especially since we have recently moved to only one guard. Well under 50 bytes/second for at most 30 minutes.
I still have a few questions, though, which is why I put so many people in Cc to this ticket. I will put my questions in the first comment.Tor: 0.3.1.x-finalMike PerryMike Perryhttps://gitlab.torproject.org/legacy/trac/-/issues/17170documenation for BandwidthRate etc should mention TCP/IP overhead not included2020-06-13T14:49:31Zstarlightdocumenation for BandwidthRate etc should mention TCP/IP overhead not includedThe Tor manual page should mention that
`BandwidthRate`, `BandwidthBurst`, etc control
only application layer byte rates and that
TCP/IP overhead adds approximately 17%
to the total, or stated a different way
the payload data rate is abo...The Tor manual page should mention that
`BandwidthRate`, `BandwidthBurst`, etc control
only application layer byte rates and that
TCP/IP overhead adds approximately 17%
to the total, or stated a different way
the payload data rate is about 15% of
total traffic.
Is easy to overlook this important
detail and it matters greatly when one is
configuring maximums in order to stay
within ISP bandwidth allocations. ISPs
count total traffic and over-limit charges
can be extortionist. Is unpleasant
to exceed them by accident.
The overhead accounting also matters if one
is setting BandwidthRate for an unmetered
Internet connection with the idea of
minimizing TCP/IP congestion in preference
to Tor router flow control.
Generally ISPs do not charge for MAC layer
bytes which are not carried on WAN links
and so the above estimates omit it.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/17592Clean up connection timeout logic2020-06-13T16:03:44ZMike PerryClean up connection timeout logicIn #6799, it was decided to keep TLS connections open for a random amount of time after they are idle, to defend against an attack that used internal Tor network connectivity information to determine Guard nodes (Slides: https://www.cryp...In #6799, it was decided to keep TLS connections open for a random amount of time after they are idle, to defend against an attack that used internal Tor network connectivity information to determine Guard nodes (Slides: https://www.cryptolux.org/images/8/85/ESORICS-2012-Presentation-2.pdf Paper: https://eprint.iacr.org/2012/432.pdf).
Unfortunately, this logic (in connection_or_set_canonical()) is kind of a mess. Relays and clients are treated the same, and client connections are also kept alive for an additional hour by predictive circuit building even when otherwise idle, where as relays are not.
If we treat relays and clients separately for this timeout, we could reduce total client keep-alive time significantly (down to 30 minutes or so), and significantly increase relay connection lifetime. This should result in reduced total connection counts on relays, with better defenses against Torscan.
This is also needed in order to put reasonable bounds on padding overhead in #16861 for mobile clients. Furthermore, aside from the wieners running middle relays behind junky home routers who will whine about connection overflow, having a more well-connected Tor network is a good idea for many reasons (not just Torscan).Tor: 0.3.1.x-finalMike PerryMike Perryhttps://gitlab.torproject.org/legacy/trac/-/issues/17604Try to use only one canonical connection2020-06-13T15:53:07ZMike PerryTry to use only one canonical connectionFor #16861, I would like to experiment with padding between relays, as well as generally keep relay-to-relay orconns open much longer. This will be beneficial against attacks like Torscan (https://eprint.iacr.org/2012/432.pdf), as well a...For #16861, I would like to experiment with padding between relays, as well as generally keep relay-to-relay orconns open much longer. This will be beneficial against attacks like Torscan (https://eprint.iacr.org/2012/432.pdf), as well as related netflow-based attacks that attempt to determine the guard node of a connection by using netflow data to accomplish the same thing as the Torscan attack.
Unfortunately, the logic for preferring orconns (is_canonical and channel_is_better()) is a mind-bender, but it appears to be the case that we may have situations where multiple orconns can be opened between relays where each side disagrees over which connection is canonical. This can happen because the source port won't match the orport in the descriptor of the remote relay for incoming connections. It will also be the case for nodes that make outgoing connections from different IP address than those in their descriptor.
I asked on #tor-dev, and two relay operators reported cases of such multiple connections to relays.
I think the following changes will improve the situation:
1. Mark all authenticated connections as canonical (everything with link proto v3 and higher will authenticate, yes?)
2. Alter channel_is_better() to prefer older orconns in the case of multiple canonical connections, and use the orconn with more circuits on it in case of age ties.
I think age is more important than number of circuits because we want to avoid giving an attacker the ability to switch an orconn between relays by creating a new one and opening a bunch of circuits on it. channel_is_better() is doing the opposite of this behavior right now, on both fronts.Tor: 0.3.1.x-finalMike PerryMike Perryhttps://gitlab.torproject.org/legacy/trac/-/issues/17857Create a consensus param to disable (netflow) padding if RSOS is enabled2020-06-13T15:10:29ZteorCreate a consensus param to disable (netflow) padding if RSOS is enabledBecause RSOS and Proposal 252 both appear to cause an onion service to make a lot of orconns to many relays, the total overhead of the netflow padding defense will be much larger for these services.
In an ideal world, we'd find some way...Because RSOS and Proposal 252 both appear to cause an onion service to make a lot of orconns to many relays, the total overhead of the netflow padding defense will be much larger for these services.
In an ideal world, we'd find some way to minimize the number of orconns that these services make, because the increased level of connection multiplexing would be better against traffic analysis attacks, and thus also mean less overhead for padding defenses (including netflow padding, but beyond that as well).
In the meantime, however, we should provide the ability to disable netflow padding via the consensus for these services. With a consensus param, we can monitor the netflow padding overhead in relay extra-info descriptors, and experiment with turning padding for RSOS/SOS on and off while observing the change in total overhead at relays.Tor: 0.3.1.x-finalMike PerryMike Perryhttps://gitlab.torproject.org/legacy/trac/-/issues/17868base64_decode_nopad() destination buffer length problem2020-06-13T14:59:07ZDavid Gouletdgoulet@torproject.orgbase64_decode_nopad() destination buffer length problemTL;DR; the `base64_decode_nopad()` doesn't work.
Here is a concrete example. We have `40 bytes` of binary data that we want to encode. With padding, that is using `base64_encode()` we end up with a size of `56 bytes`. Those resulting by...TL;DR; the `base64_decode_nopad()` doesn't work.
Here is a concrete example. We have `40 bytes` of binary data that we want to encode. With padding, that is using `base64_encode()` we end up with a size of `56 bytes`. Those resulting bytes, when passed to `base64_decode()`, the check on the destination buffer done in that function makes it that we need `42 bytes` and not the original `40 bytes`. This is due because of the padding.
One solution, instead of explicitly adding 2 bytes like it's been done in many places in the code, it is to use the `_nopad()` interface. However, the `base64_decode_nopad()` simply adds some `=` at the end with a new source buffer and passes along the base64_decode() function. However the `dstlen` that is the destination buffer length where the decoded data will go is not updated to reflect the new length of the source buffer so the call fails because of the `dstlen` check in the decode function.
Passing 40 bytes for `dstlen` and 54 for `srclen` (which is the expected value _without_ padding), the nopad() call changes `srclen` to 56 bytes but then `dstlen` should be 42 bytes else the call fails.
I'm not sure how to fix that properly apart from making `_nopad()` call allocating a new source buffer if needed. I would much prefer that than the caller adding bytes beforehand making the code cryptic and honestly unsafe to any future length changes.
Thoughts?Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/18054prop224: Decide how to proceed with torrc options2020-06-13T14:53:26ZDavid Gouletdgoulet@torproject.orgprop224: Decide how to proceed with torrc optionsWith next-gen HS, we'll have both new and legacy system running in parallel. However, we should encourage as much as possible the adoption of the new system when setting up a new hidden service.
From a torrc perspective, we should proba...With next-gen HS, we'll have both new and legacy system running in parallel. However, we should encourage as much as possible the adoption of the new system when setting up a new hidden service.
From a torrc perspective, we should probably use the same options as we have now but making tor code a bit more smarter when parsing them. One idea would be to add an option that indicates if the user wants the legacy protocol or not:
Something that could look like:
```
HiddenServiceLegacy 1
HiddenServiceDir <PATH>
HiddenServicePort ...
```
And `HiddenServiceLegacy` by default would be 0. This option should be deprecated over time.
Another possibility is NOT having this config option and only if "private_key" file exists in the `HiddenServiceDir` and it's `RSA1024`, we switch to legacy. Without it, we go next-gen. So in other words, we DO NOT let a user setup a legacy HS unless she has a key for it placed in the HS directory. This will allow current HS setup to still work after upgrade and not make operators add an extra option.
Basically, the key type would be our heuristic to switch from legacy to next-gen.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/18329Let bridges indicate when they don't want BridgeDB to distribute their address2020-06-13T18:28:59ZKarsten LoesingLet bridges indicate when they don't want BridgeDB to distribute their addressRight now, bridges can decide whether they want to be a public bridge that gets distributed via BridgeDB or a private bridge that only gets used by clients who learn its address via some other, private channel. The default is that a bri...Right now, bridges can decide whether they want to be a public bridge that gets distributed via BridgeDB or a private bridge that only gets used by clients who learn its address via some other, private channel. The default is that a bridge is a public bridges, unless it sets `PublishServerDescriptor 0` in its `torrc` file. This works fine with respect to BridgeDB not distributing private bridges. But a lesser known problem is that a bridge that doesn't publish its descriptor also does not contribute to bridge usage statistics on Metrics that are based on bridge extra-info descriptors.
The major use case that comes to mind is a bundled bridge whose address is shipped together with Tor Browser or another application. In the past we tried to remind operators of these bridges to also publish descriptors, so that their statistics are included on Metrics. But it turns out that some censors, who carefully scrape bridge addresses from BridgeDB, do not extract bridge addresses from the various bundles. Still, bundled bridges see a large number of bridge users and we should really include them in the statistics.
Another use case could be private bridges that somebody sets up for themselves and their friends. Maybe these operators would be fine contributing to the statistics if that doesn't automatically mean they need to share their bridge with other users.
I think this feature is relatively easy to build. We would need:
- a new descriptor line "bridgedb off", or something even more intuitive and extensible, that tells BridgeDB that this bridge's address should not be distributed,
- a new torrc option or extension of an existing option, maybe "PublishServerDescriptor bridge-auth" or, again, something more intuitive, to include the line above in the descriptor, and
- an extension of BridgeDB to ignore bridges with this line when parsing descriptors.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/18859A new SOCKS connection should use a pre-built circuit for its first stream2020-06-13T15:20:49ZArthur EdelsteinA new SOCKS connection should use a pre-built circuit for its first streamSince #3455, we use SOCKS auth isolation in Tor Browser to separate URL bar domains to different tor circuits. When the user browses to a new URL bar domain, a new SOCKS connection is opened with a SOCKS username/password unique to the s...Since #3455, we use SOCKS auth isolation in Tor Browser to separate URL bar domains to different tor circuits. When the user browses to a new URL bar domain, a new SOCKS connection is opened with a SOCKS username/password unique to the site's domain.
By telneting to the tor control port, I observed that immediately after I entered a new URL bar domain in a Tor Browser tab, a new circuit was built and assigned the SOCK_USERNAME and SOCKS_PASSWORD for that URL bar domain.
It seems there would be better performance if we could use an existing, pre-built (yet-unused) circuit when a new SOCKS connection opens, and assign the SOCKS_USERNAME and SOCKS_PASSWORD to the pre-built circuit. That way the user wouldn't have to wait for a circuit to be established after requesting a new website.
I don't know yet whether this is something that can be adjusted by config settings or if we would need to patch tor somehow.Tor: 0.3.1.x-finalArthur EdelsteinArthur Edelsteinhttps://gitlab.torproject.org/legacy/trac/-/issues/19418i2d_RSAPublicKey retval ignored in multiple callsites2020-06-13T14:58:40ZGeorge Kadianakisi2d_RSAPublicKey retval ignored in multiple callsitesHello. Here follows a bug report by `Guido Vranken` received through the hackerone program.
There are various places in the codebase where we don't check the retval of `i2d_RSA_PublicKey()` (or its callers). That function can fail in ca...Hello. Here follows a bug report by `Guido Vranken` received through the hackerone program.
There are various places in the codebase where we don't check the retval of `i2d_RSA_PublicKey()` (or its callers). That function can fail in cases of OOM, and this is something we should be handling correctly. This is a similar case to #17686.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/19564SR: use macros to compute our base64 encoded length2020-06-13T14:59:16ZDavid Gouletdgoulet@torproject.orgSR: use macros to compute our base64 encoded lengthTicket #19531 once done should be adding some macros to compute the length of encoded base64 string.
Let's use them for these in `shared_random.h`:
- `SR_COMMIT_BASE64_LEN`
- `SR_REVEAL_BASE64_LEN`
- `SR_SRV_VALUE_BASE64_LEN`Ticket #19531 once done should be adding some macros to compute the length of encoded base64 string.
Let's use them for these in `shared_random.h`:
- `SR_COMMIT_BASE64_LEN`
- `SR_REVEAL_BASE64_LEN`
- `SR_SRV_VALUE_BASE64_LEN`Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/19824/var/run/tor/control socket not created because of /var/run/tor permission issue2020-06-13T14:59:54Zadrelanos/var/run/tor/control socket not created because of /var/run/tor permission issueUsing Tor `0.2.8.6` from deb.torproject.org, `/var/run/tor/control` is no longer created because of a permission issue. As a manual workaround, `sudo chmod --recursive 700 /var/run/tor` works.
The symptom in Tor's log is the following:
...Using Tor `0.2.8.6` from deb.torproject.org, `/var/run/tor/control` is no longer created because of a permission issue. As a manual workaround, `sudo chmod --recursive 700 /var/run/tor` works.
The symptom in Tor's log is the following:
```
Aug 03 17:36:33.000 [warn] Permissions on directory /var/run/tor are too permissive.
```
Rather than `755` Tor's `/lib/systemd/system/tor@default.service` should use `700`. I.e. rather than using:
```
ExecStartPre=/usr/bin/install -Z -m 02755 -o debian-tor -g debian-tor -d /var/run/tor
```
`/lib/systemd/system/tor@default.service` should use:
```
ExecStartPre=/usr/bin/install -Z -m 02700 -o debian-tor -g debian-tor -d /var/run/tor
```Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/19966torproxy goes south with more than 1 connection attempt per second to a hidde...2020-06-13T15:00:28ZTractorproxy goes south with more than 1 connection attempt per second to a hidden serviceI have a web service running as a tor hidden service. I have a test script that connects to the hidden service through tor proxy, sends a short string, gets a short reply, and then the hidden service closes the connection. Each time th...I have a web service running as a tor hidden service. I have a test script that connects to the hidden service through tor proxy, sends a short string, gets a short reply, and then the hidden service closes the connection. Each time the test script connects, it uses a different socks username, so it should get a fresh tor circuit each time.
This works mostly ok with Tor v0.2.7.6 (git-7a489a6389110120) running on Windows 7 with Libevent 2.0.22-stable, OpenSSL 1.0.1t and Zlib 1.2.8. With four test scripts, it makes about 1 connection per second. Obviously, with any network, it is going to get some connection and network errors, but it recovers eventually.
With Tor v0.2.8.6 (git-4d217548e3f05569) running on Windows 7 with Libevent 2.0.22-stable, OpenSSL 1.0.1t and Zlib 1.2.8, it doesn't run so well. Up to about one connection every two seconds, it runs ok, but as soon as I go above that, tor proxy goes south and starts refusing all connections and the logging error messages shown below. This behavior makes is unsuitable/unreliable to use in a "production" environment.
It seems to not matter what version of tor I use on the hidden service side--only the version on the client side seems to matter, and that is where the errors messages are seen.
This is the contents of my torrc file, which works under v0.2.7.6 but not under v0.2.8.6.
Log notice stdout
LogMessageDomains 1
SafeLogging 0
MaxClientCircuitsPending 200
KeepalivePeriod 15
SocksTimeout 60
NewCircuitPeriod 1
LearnCircuitBuildTimeout 1
CircuitBuildTimeout 60
NumEntryGuards 20
The error messages are:
Aug 23 18:30:32.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:32.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:34.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:34.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:35.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:35.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:35.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:35.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:36.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:36.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:36.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:36.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:36.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:36.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:37.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:37.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:37.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:37.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:37.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:37.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:37.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:37.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:38.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:38.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:39.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:39.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:39.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:39.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:39.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:40.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:40.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:40.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:40.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:40.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:40.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:40.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:40.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:41.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:41.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:42.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:42.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:42.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:42.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:42.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:43.000 [warn] {REND} Fetching v2 rendezvous descriptor failed. Retrying at another directory.
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
Aug 23 18:30:43.000 [notice] {REND} Closing stream for 'vgfdgkr7tudleiq3.onion': hidden service is unavailable (try again later).
**Trac**:
**Username**: AlanTor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/20267Use -DOPENSSL_SYS_WIN32 on Windows2020-06-13T15:01:50ZteorUse -DOPENSSL_SYS_WIN32 on WindowsTor fails to compile on Windows MinGW/msys with an error about X509_NAME being an integer literal.
This is because OPENSSL_SYS_WIN32 is not defined, but it should be.
When we are on Win32, we should define this preprocessor directive.Tor fails to compile on Windows MinGW/msys with an error about X509_NAME being an integer literal.
This is because OPENSSL_SYS_WIN32 is not defined, but it should be.
When we are on Win32, we should define this preprocessor directive.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/20341torrc man page missing components for Bridge line2020-06-13T15:02:01ZTractorrc man page missing components for Bridge lineThe man page for torrc Bridge currently has an entry:
Bridge [transport] IP:ORPort [fingerprint]
an example line i was just given had a bit more than this:
Bridge [transport] IP:ORPort [fp] [cert=...] ] iat-mode=0
perhaps a descripti...The man page for torrc Bridge currently has an entry:
Bridge [transport] IP:ORPort [fingerprint]
an example line i was just given had a bit more than this:
Bridge [transport] IP:ORPort [fp] [cert=...] ] iat-mode=0
perhaps a description of iat-mode and its values would be helpful? and extend the config line?
**Trac**:
**Username**: stefanibTor: 0.3.1.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/legacy/trac/-/issues/20458Integration tests should be run locally before committing code changes2020-06-13T15:02:19ZChelsea KomloIntegration tests should be run locally before committing code changesCurrently, it is easy to submit a patch for code changes without first running integration tests locally.
Two changes could help with this:
1. Adding this information to documentation for contributors
2. Adding a make task that runs all...Currently, it is easy to submit a patch for code changes without first running integration tests locally.
Two changes could help with this:
1. Adding this information to documentation for contributors
2. Adding a make task that runs all necessary tasks before contributing new code, which are:
`make check-spaces`
`make check`
`make test-network-all`
We should investigate how to handle:
- Someone not having have chutney
- Someone not having the necessary network connection for integration tests to succeed
- If chutney tests fail because of flakiness (race conditions for example) rather than legitimate failures.Tor: 0.3.1.x-finalChelsea KomloChelsea Komlohttps://gitlab.torproject.org/legacy/trac/-/issues/20509Directory authorities should reject relays with #20499 bug2020-07-31T12:44:02ZRoger DingledineDirectory authorities should reject relays with #20499 bugOnce we resolve #20499, or more precisely, once we figure out which versions it affects, we should teach the directory authorities to withhold the Guard flag from relays that are running those versions.
Right now you can't bootstrap fro...Once we resolve #20499, or more precisely, once we figure out which versions it affects, we should teach the directory authorities to withhold the Guard flag from relays that are running those versions.
Right now you can't bootstrap from e.g. sofia, which is a huge guard running 0.2.9.3-alpha.
Then later -- or maybe this will turn out to be the easiest solution for everything -- we can teach the directory authorities to simply reject descriptors from relays running these buggy versions.
[I'm not sure which milestone to put this in, since it will need to get into a majority of directory authorities for it to matter. :/ ]Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/20571When we are really satisfied that it is right, tell protover.c about prop224 ...2020-06-13T15:02:57ZNick MathewsonWhen we are really satisfied that it is right, tell protover.c about prop224 HSDir supportThis will be trivial to do, but nontrivial to know when we should do it.
prop224-supporting hidden services and relays will need to know which Tor relays they should use to upload and download the new kind of descriptors. So, they shoul...This will be trivial to do, but nontrivial to know when we should do it.
prop224-supporting hidden services and relays will need to know which Tor relays they should use to upload and download the new kind of descriptors. So, they should use the preferred mechanism for it.
As of proposal 264, that's protocol subversions.
We should only advertise support for this server-side once we're pretty sure we aren't changing the server side any more.Tor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/20908Display the fingerprint when downloading consensuses from fallbacks2020-06-13T15:04:06ZteorDisplay the fingerprint when downloading consensuses from fallbacksJust using this for the bug number, will be fixed in #18828.Just using this for the bug number, will be fixed in #18828.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/20913Increase fallback stability, flag percentage, and bandwidth2020-06-13T16:05:54ZteorIncrease fallback stability, flag percentage, and bandwidthIn #18828, I had to decrease the fallback criteria to get ~160 fallbacks.
It would be great to increase these to:
* Stability: 3 months+ (from 7 days)
* Flags: 95%+ (from 90%)
* Bandwidth: 2MBytes/s+ (from 1MByte/s)In #18828, I had to decrease the fallback criteria to get ~160 fallbacks.
It would be great to increase these to:
* Stability: 3 months+ (from 7 days)
* Flags: 95%+ (from 90%)
* Bandwidth: 2MBytes/s+ (from 1MByte/s)Tor: 0.3.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20988Test fgets_eagain fails on FreeBSD-amd642020-06-13T15:13:26ZcypherpunksTest fgets_eagain fails on FreeBSD-amd64According to the [BSD Buildbot](https://buildbot.pixelminers.net/builders)
```
util/fgets_eagain:
FAIL src/test/test_util.c:3952: assert(retptr OP_EQ buf): 0x0 vs 0x7fffffffe944
[fgets_eagain FAILED]
```
This means that fgets retur...According to the [BSD Buildbot](https://buildbot.pixelminers.net/builders)
```
util/fgets_eagain:
FAIL src/test/test_util.c:3952: assert(retptr OP_EQ buf): 0x0 vs 0x7fffffffe944
[fgets_eagain FAILED]
```
This means that fgets returns a null pointer on partial lines instead of the buffer as expected.
Previously this test was passing but started failing with [build #105](https://buildbot.pixelminers.net/builders/FreeBSD-amd64/builds/105). Looking at the changes to libc it looks like this is caused by [revision 305413](https://svnweb.freebsd.org/base/head/lib/libc/stdio/fgets.c?r1=305413&r2=305412&pathrev=305413) (which was added earlier in the same month the test started failing).
I'm unsure whether FreeBSD is right and other libc implementations are wrong or the other way around.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21008hs: Remove the introduction point private key material from hs_descriptor.h2020-06-13T15:04:36ZDavid Gouletdgoulet@torproject.orghs: Remove the introduction point private key material from hs_descriptor.hWe need to change the `curve25519_keypair_t` from `hs_desc_intro_point_t` to a public key only because both client and service will use that field.We need to change the `curve25519_keypair_t` from `hs_desc_intro_point_t` to a public key only because both client and service will use that field.Tor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21028tor never checks TLS certificate lifetimes2020-06-13T15:04:42Zteortor never checks TLS certificate lifetimesWe have tor_tls_check_lifetime(), why don't we use it?
This could be an issue if a relay never rotates its certificate, or its certificate is out of date because its clock is wrong.We have tor_tls_check_lifetime(), why don't we use it?
This could be an issue if a relay never rotates its certificate, or its certificate is out of date because its clock is wrong.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21056Could not pick one of the responsible hidden service directories, because we ...2020-06-13T15:04:52ZTracCould not pick one of the responsible hidden service directories, because we requested them all recently without success.I am seeing connections to hidden services reliably fail, with this in the log:
Dec 21 13:50:16.000 [info] connection_ap_handshake_rewrite_and_attach(): Got a hidden service request for ID '[scrubbed]'
Dec 21 13:50:16.000 [info] connect...I am seeing connections to hidden services reliably fail, with this in the log:
Dec 21 13:50:16.000 [info] connection_ap_handshake_rewrite_and_attach(): Got a hidden service request for ID '[scrubbed]'
Dec 21 13:50:16.000 [info] connection_ap_handshake_rewrite_and_attach(): Unknown descriptor [scrubbed]. Fetching.
Dec 21 13:50:16.000 [debug] rend_client_refetch_v2_renddesc(): Fetching v2 rendezvous descriptor for service [scrubbed]
Dec 21 13:50:16.000 [info] pick_hsdir(): Could not pick one of the responsible hidden service directories, because we requested them all recently without success.
Dec 21 13:50:16.000 [info] pick_hsdir(): Could not pick one of the responsible hidden service directories, because we requested them all recently without success.
Dec 21 13:50:16.000 [info] fetch_v2_desc_by_addr(): Could not pick one of the responsible hidden service directories to fetch descriptors, because we already tried them all unsuccessfully.
In my test case, I have 2 hidden services running on a machine. Both have been started up for the first time in the past 5-10 minutes. On the same machine, I open concurrently two socks connections, one to each hidden service. Reiably, one of the socks connections succeeds, and the other fails. After it starts failing, retries using that onion address continue to fail for several minutes.
Kind of looks like one socks request is breaking the other one. This seems similar to #16501 and #15937 but I am pretty sure I am only making 2 socks connections, to two different onion addresses.
Attached log shows this occurring, and then after a while the bad state cleared and a retry to connect to the hidden service that it had not been able to resolve succeeded.
**Trac**:
**Username**: joeyhTor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21120evdns_get_orig_address: tor_fragile_assert() warning for unknown rtype.2020-06-13T15:05:06ZTracevdns_get_orig_address: tor_fragile_assert() warning for unknown rtype.The trace (below) appeared while routinely examining my tor log while having network connectivity issues (arch is x86/32 bit, kernel is 3.11):
```
Jan 02 16:22:56.000 [warn] {BUG} tor_bug_occurred_(): Bug: src/or/dnsserv.c:294: evdns_ge...The trace (below) appeared while routinely examining my tor log while having network connectivity issues (arch is x86/32 bit, kernel is 3.11):
```
Jan 02 16:22:56.000 [warn] {BUG} tor_bug_occurred_(): Bug: src/or/dnsserv.c:294: evdns_get_orig_address: This line should not have been reached. (Future instances of this warning will be silenced.) (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: Line unexpectedly reached at evdns_get_orig_address at src/or/dnsserv.c:294. Stack trace: (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(log_backtrace+0x56) [0xb765aee6] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(tor_bug_occurred_+0xf5) [0xb7676435] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(dnsserv_resolved+0x255) [0xb7642435] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(connection_ap_handshake_socks_resolved+0x8e) [0xb75fe4fe] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(+0x593fe) [0xb75513fe] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(+0x5d296) [0xb7555296] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(circuit_receive_relay_cell+0x316) [0xb7556f66] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(command_process_cell+0x22f) [0xb75d8f0f] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(channel_queue_cell+0xd2) [0xb75b0882] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(channel_tls_handle_cell+0x2bc) [0xb75b695c] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(+0x11095b) [0xb760895b] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(+0xf8df4) [0xb75f0df4] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(connection_handle_read+0x7e3) [0xb75f9983] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(+0x3458f) [0xb752c58f] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/lib/libevent-2.1.so.6(+0x207d5) [0xb745a7d5] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/lib/libevent-2.1.so.6(event_base_loop+0x2b4) [0xb745af84] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(do_main_loop+0x46c) [0xb752732c] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(tor_main+0x1315) [0xb7528965] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(main+0x33) [0xb7523e23] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /lib/libc.so.6(__libc_start_main+0xe6) [0xb7033cc6] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
Jan 02 16:22:56.000 [warn] {BUG} Bug: /usr/bin/tor(+0x2bd11) [0xb7523d11] (on Tor 0.2.9.4-alpha 8b0755c9bb296ae2)
```
**Trac**:
**Username**: mr-4Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21121Update fallback whitelist2020-06-13T16:06:00ZteorUpdate fallback whitelistRelay operators have emailed me with updates.
I'm tracking them in my github branch fallbacks-201701.Relay operators have emailed me with updates.
I'm tracking them in my github branch fallbacks-201701.Tor: 0.3.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/21151man page lists wrong default for DataDirectory2020-06-13T15:05:15ZRoger Dingledineman page lists wrong default for DataDirectoryIn my doc/tor.1 in my git checkout, I have
```
DataDirectory DIR
Store working data in DIR (Default: /usr/local/var/lib/tor)
```
Apparently the underlying code (in tor.1.txt) is
```
[[DataDirectory]] **DataDirectory** ...In my doc/tor.1 in my git checkout, I have
```
DataDirectory DIR
Store working data in DIR (Default: /usr/local/var/lib/tor)
```
Apparently the underlying code (in tor.1.txt) is
```
[[DataDirectory]] **DataDirectory** __DIR__::
Store working data in DIR (Default: @LOCALSTATEDIR@/lib/tor)
```
It looks like if DataDirectory remains unset (which is the default), on Windows you get
```
get_windows_conf_root()
```
whereas on other platforms, you get
```
d = "~/.tor";
```
and then it's only if you're running as root that you get
```
fn = tor_strdup(LOCALSTATEDIR PATH_SEPARATOR "tor");
```
But I admit that all of this is confusing, because we have functions like `find_torrc_filename()` and `get_default_conf_file()` and `get_torrc_fname()` so it's hard to say for sure.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21155Similar to #14917: Client's choice of rend point can leak info about guard(s)...2020-06-13T15:05:16ZJaymSimilar to #14917: Client's choice of rend point can leak info about guard(s) of misconfigured hidden services with EntryNodes optionHello !
I discovered #14917 while configuring an onion service with the EntryNodes option set. I believe (after checking the tor-0.2.9.8 source code) that a similar problem arises when the EntryNodes option is set AND the operator confi...Hello !
I discovered #14917 while configuring an onion service with the EntryNodes option set. I believe (after checking the tor-0.2.9.8 source code) that a similar problem arises when the EntryNodes option is set AND the operator configures entry nodes that are part of the same family or the same /16. (let's say that the operator configures the service with 2 of its own guard nodes running in the same cloud provider, thinking this move is wise). Then this happens:
- When someone use a RDV point of the same family or the same /16 than the onion's guards, then as you said: "entry_list_is_constrained() is true, so populate_live_entry_guards() will happily return an empty list if your one choice is inappropriate, resulting in choose_random_entry_impl() returning NULL".
Is there a reason why we do not check family, /16 and user misconfiguration ? "EntryNodes fingerprint1, fingerprint1" works just fine for example. What do you think ?Tor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21210Analyze, and maybe improve, consensus diff proposal2020-06-13T15:05:24ZNick MathewsonAnalyze, and maybe improve, consensus diff proposalWe should use stats to re-run our numbers on the consensus diff proposal (140), and see how much bandwidth we expect to save. We should consider the impact of this proposal alongside alternative or related proposals, such as ones that w...We should use stats to re-run our numbers on the consensus diff proposal (140), and see how much bandwidth we expect to save. We should consider the impact of this proposal alongside alternative or related proposals, such as ones that would cause clients to download the consensus less frequently.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21211Write and analyze proposals for compressing consensus (diff)s with better alg...2020-06-13T15:05:24ZNick MathewsonWrite and analyze proposals for compressing consensus (diff)s with better algorithms**The idea:** Consensus documents are compressed with zlib, but nobody has to compress any given consensus more than once. Therefore, we can safely use more CPU compressing them, and save bandwidth on consensus downloads by switching to...**The idea:** Consensus documents are compressed with zlib, but nobody has to compress any given consensus more than once. Therefore, we can safely use more CPU compressing them, and save bandwidth on consensus downloads by switching to something else instead of zlib for consensuses.
This same analysis also applies to consensus diffs.
For this ticket, we should look at the code complexity and potential bandwidth savings here, and decide whether they are worth it.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21293circuit_receive_relay_cell(): Bug: relay crypt failed. Dropping connection.2020-06-13T15:05:38Zs7rcircuit_receive_relay_cell(): Bug: relay crypt failed. Dropping connection.I've hit this today on a machine running Debian Jessie and Tor `0.3.0.1-alpha-dev (git-1a45398ffa713ca3+5156f73)` acting as an onion service client with a SocksPort open and used as well. Functionality appears not to be affected, everyth...I've hit this today on a machine running Debian Jessie and Tor `0.3.0.1-alpha-dev (git-1a45398ffa713ca3+5156f73)` acting as an onion service client with a SocksPort open and used as well. Functionality appears not to be affected, everything continues to run normally.
Besides the incoming rendezvous traffic, this instance also sends outgoing traffic via the SocksPort but to .onion destinations only, so it's mostly rendezvous traffic.
```
Jan 23 15:31:44.000 [warn] circuit_receive_relay_cell(): Bug: relay crypt failed. Dropping connection. (on Tor 0.3.0.1-alpha-dev )
```Tor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21329onions/detached and onions/current should return empty lists instead of errors2017-05-15T19:44:31Zmeejahonions/detached and onions/current should return empty lists instead of errorsIf you do a GETINFO query on `onions/detached` or `onions/current` and there aren't any, a 551 error is returned. Most other things just return an empty list/value in this case and I think these should too.
One consequence of this is th...If you do a GETINFO query on `onions/detached` or `onions/current` and there aren't any, a 551 error is returned. Most other things just return an empty list/value in this case and I think these should too.
One consequence of this is that if you ask for `onions/current` with some other keys and there are no current onions, the whole query will fail.
So `GETINFO onions/current` should return `onions/current=` instead of an error.
For example you can try `carml cmd getinfo orconn-status onions/current` to see this behavior.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21334prop224: Update prop224 HS desriptor generation code to produce the latest de...2020-06-13T15:07:14ZGeorge Kadianakisprop224: Update prop224 HS desriptor generation code to produce the latest descriptor formatThis is the older cousin of #20852. In this ticket we need to go over our prop224 HS descriptor generation code and make it produce descriptors that conform with the latest format (which includes the fake client authorization entries and...This is the older cousin of #20852. In this ticket we need to go over our prop224 HS descriptor generation code and make it produce descriptors that conform with the latest format (which includes the fake client authorization entries and such).Tor: 0.3.1.x-finalGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/legacy/trac/-/issues/21406The channel is_client flag is inaccurate2020-06-13T15:08:15ZteorThe channel is_client flag is inaccurateThe channel_t is_client flag is inaccurate: relays set it when the other end uses a CREATE_FAST cell, but usecreatefast is set to 0 in the consensus.
This means that the only time CREATE_FAST is used is when a relay gets an extend reque...The channel_t is_client flag is inaccurate: relays set it when the other end uses a CREATE_FAST cell, but usecreatefast is set to 0 in the consensus.
This means that the only time CREATE_FAST is used is when a relay gets an extend request *without* an ntor key, and the purpose of the circuit is *not* one of the hidden service purposes where TAP is allowed.
See should_use_create_fast_for_circuit() for details.Tor: 0.3.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/21407Make the usecreatefast default 0 in tor to match the consensus2020-06-13T15:06:06ZteorMake the usecreatefast default 0 in tor to match the consensusIn #9386 we made clients use CREATE_FAST less, and changed the value in the consensus to 0. But we still use the default of 1 when bootstrapping (because there is no consensus).
We should change this default to 0.
Sticking this in 0.3....In #9386 we made clients use CREATE_FAST less, and changed the value in the consensus to 0. But we still use the default of 1 when bootstrapping (because there is no consensus).
We should change this default to 0.
Sticking this in 0.3.1 because it's a security hole for the reasons mentioned in #9386. (I'd go for 0.3.0, but that's frozen, and this change could introduce bootstrapping bugs.)Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21439Add a configure option to disable safety features that make fuzzing harder2020-06-13T15:06:14ZNick MathewsonAdd a configure option to disable safety features that make fuzzing harderWe've got quite a few places in our code where we use redundant safety features to prevent bugs from turning into really serious bugs. But many of those safety features interfere with fuzzing, by covering up any underlying bugs that fuz...We've got quite a few places in our code where we use redundant safety features to prevent bugs from turning into really serious bugs. But many of those safety features interfere with fuzzing, by covering up any underlying bugs that fuzzing would otherwise detect.
For example, I'm thinking of:
* The 4-byte sentinel word at the end of each buffer chunk
* Various places in our code where we NUL-terminate stuff that doesn't actually (we hope!) need to be NUL-terminated.
* The entire "memarea" fragmentation-resistant allocation strategy.
* Probably some other stuff too
But in addition to hardening our code a little, these features all make some classes of memory bug less likely to get noticed by the sanitizers.
Now, you might argue that there's no need to have a way to fuzz without those safety features, if they actually do provide safety. But on the other hand, they're meant to provide _redundant_ safety, and if they are ever actually needed, that's a bug in our code that we ought to fix.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21470Write unit tests for security regressions2020-06-13T15:06:20ZteorWrite unit tests for security regressionsWhen we fix a security vulnerability like the ones in [
https://trac.torproject.org/projects/tor/wiki/TROVE TROVE], we sometimes forget to include a unit test.
Some security vulnerabilities are easy to test, such as parsing errors. We s...When we fix a security vulnerability like the ones in [
https://trac.torproject.org/projects/tor/wiki/TROVE TROVE], we sometimes forget to include a unit test.
Some security vulnerabilities are easy to test, such as parsing errors. We should write tests that fail with the bug, but succeed when it is fixed.
Here's an initial list of security fixes we can write unit tests for:
* #20894
* #21018
* #21278 (and #21450)
Let's open a child ticket of this ticket for each one.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21496Check string passed to extrainfo_parse_entry_from_string2020-06-13T15:06:27ZteorCheck string passed to extrainfo_parse_entry_from_stringscan-build reports the following out of bounds access, because it doesn't know that s is always non-NULL. But we should guard against it being NULL.
extrainfo_parse_entry_from_string:
```
+ if (BUG(!s))
+ return;
while (end > s+2...scan-build reports the following out of bounds access, because it doesn't know that s is always non-NULL. But we should guard against it being NULL.
extrainfo_parse_entry_from_string:
```
+ if (BUG(!s))
+ return;
while (end > s+2 && *(end-1) == '\n' && *(end-2) == '\n')
```Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21510Make unit test captured log messages consistent2020-06-13T15:06:32ZteorMake unit test captured log messages consistentThere's an extra colon and missing space in b0a9e54705d.
It would be nice to clean these up - there's no reason to backport.There's an extra colon and missing space in b0a9e54705d.
It would be nice to clean these up - there's no reason to backport.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21529Bootstrap fails if one pinned ExitNode is not running2020-06-13T15:06:37ZDavid Gouletdgoulet@torproject.orgBootstrap fails if one pinned ExitNode is not runningSome user reported that he/she pinned a single ExitNode and that relay is not running which ultimately make tor not bootstrap. (Taken from #21520)
```
Feb 22 00:08:35.018 [notice] Opening Socks listener on 127.0.0.1:9550
Feb 22 00:08:35...Some user reported that he/she pinned a single ExitNode and that relay is not running which ultimately make tor not bootstrap. (Taken from #21520)
```
Feb 22 00:08:35.018 [notice] Opening Socks listener on 127.0.0.1:9550
Feb 22 00:08:35.018 [notice] Opening Control listener on 127.0.0.1:9551
Feb 22 00:08:35.000 [notice] Bootstrapped 0%: Starting
Feb 22 00:08:36.000 [notice] Bootstrapped 45%: Asking for relay descriptors
Feb 22 00:08:38.000 [notice] Bootstrapped 50%: Loading relay descriptors
Feb 22 00:08:46.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 7134/7268, and can only build 0% of likely paths. (We have 98% of guards bw, 98% of midpoint bw, and 0% of exit bw = 0% of path bw.)
```
This seems peculiar behavior because Tor should be able to download the consensus nevertheless through Dir Auth. or Fallback Directories? A tor without a working ExitNode is a valid Tor (hidden service client for instance) so it should bootstrap.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21540Warning "Failure from drain_fd"2020-06-13T15:06:39ZTracWarning "Failure from drain_fd" "Failure from drain_fd: No error [x similar messages suppressed in 72000
seconds]"
"x" stands for any number
**Possible the same as #16992**
```
Feb 18 19:41:59.564 [Warnung] Failure from drain_fd: No error [5 similar message(s) sup... "Failure from drain_fd: No error [x similar messages suppressed in 72000
seconds]"
"x" stands for any number
**Possible the same as #16992**
```
Feb 18 19:41:59.564 [Warnung] Failure from drain_fd: No error [5 similar message(s) suppressed in last 7200 seconds]
Feb 18 20:04:08.501 [Hinweis] Heartbeat: Tor's uptime is 16:22 hours, with 2 circuits open. I've sent 31.55 MB and received 211.71 MB.
Feb 18 20:04:08.501 [Hinweis] Average packaged cell fullness: 73.875%. TLS write overhead: 5%
Feb 18 20:04:08.501 [Hinweis] Circuit handshake stats since last time: 1/1 TAP, 0/0 NTor.
Feb 18 20:04:08.501 [Hinweis] Since startup, we have initiated 0 v1 connections, 0 v2 connections, 0 v3 connections, and 43 v4 connections; and received 3 v1 connections, 12 v2 connections, 17 v3 connections, and 701 v4 connections.
Feb 18 21:54:40.556 [Warnung] Failure from drain_fd: No error [1 similar message(s) suppressed in last 7200 seconds]
Feb 19 00:00:07.179 [Warnung] Failure from drain_fd: No error [9 similar message(s) suppressed in last 7200 seconds]
Feb 19 01:15:16.376 [Hinweis] Interrupt: we have stopped accepting new connections, and will shut down in 30 seconds. Interrupt again to exit now.
Feb 19 15:14:06.133 [Hinweis] Tor 0.2.9.9 (git-56788a2489127072) running on Windows 7 with Libevent 2.0.22-stable, OpenSSL 1.0.2j and Zlib 1.2.8.
Feb 19 15:14:06.134 [Hinweis] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Feb 19 15:14:06.134 [Hinweis] Read configuration file "C:\Program Files (x86)\Vidalia Relay Bundle\Data\Tor\torrc".
Feb 19 15:14:06.134 [Warnung] Path for DataDirectory (C:/Program Files (x86)/Vidalia Relay Bundle/Data/Tor) is relative and will resolve to C:\Program Files (x86)\Vidalia Relay Bundle\Data\Tor. Is this what you wanted?
Feb 19 15:14:06.134 [Hinweis] Based on detected system memory, MaxMemInQueues is set to 2048 MB. You can override this by setting MaxMemInQueues by hand.
Feb 19 15:14:06.134 [Hinweis] Opening Socks listener on 127.0.0.1:9050
Feb 19 15:14:06.134 [Hinweis] Opening Control listener on 127.0.0.1:9051
Feb 19 15:14:06.134 [Hinweis] Opening OR listener on 0.0.0.0:443
Feb 19 15:14:06.135 [Hinweis] Opening Directory listener on 0.0.0.0:9030
Feb 19 17:28:38.911 [Warnung] Failure from drain_fd: No error [74 similar message(s) suppressed in last 7200 seconds]
Feb 19 20:48:42.369 [Warnung] Failure from drain_fd: No error [2 similar message(s) suppressed in last 7200 seconds]
Feb 19 21:15:07.100 [Hinweis] Heartbeat: Tor's uptime is 5:59 hours, with 3 circuits open. I've sent 55.97 MB and received 935.17 MB.
Feb 19 21:15:07.101 [Hinweis] Average packaged cell fullness: 57.651%. TLS write overhead: 5%
Feb 19 21:15:07.101 [Hinweis] Circuit handshake stats since last time: 1/1 TAP, 115/115 NTor.
Feb 19 21:15:07.101 [Hinweis] Since startup, we have initiated 0 v1 connections, 0 v2 connections, 0 v3 connections, and 148 v4 connections; and received 0 v1 connections, 10 v2 connections, 11 v3 connections, and 377 v4 connections.
```
**Trac**:
**Username**: PjotrVTor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21554Inventory proposals that need merging into specs ; merge them.2020-06-13T15:06:41ZNick MathewsonInventory proposals that need merging into specs ; merge them.Before we can call 0.3.0 done, we must have the specs up-to-date.Before we can call 0.3.0 done, we must have the specs up-to-date.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21564Regenerate fallback list for 0.3.1 or 0.3.22020-06-13T16:06:01ZteorRegenerate fallback list for 0.3.1 or 0.3.2This involves:
* checking for new relays
* contacting potential fallback operators
* rebuilding the list
* contacting all fallback operators
See detailed instructions here:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallb...This involves:
* checking for new relays
* contacting potential fallback operators
* rebuilding the list
* contacting all fallback operators
See detailed instructions here:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrorsTor: 0.3.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/21580circuit_build_needed_circs comment mentions router_have_min_dir_info, which d...2020-06-13T15:06:46Zteorcircuit_build_needed_circs comment mentions router_have_min_dir_info, which does not existThis was introduced somewhere in 0.3.0 with the new guard code.This was introduced somewhere in 0.3.0 with the new guard code.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21586Assertion my_rsa_cert failed in sr_generate_our_commit when BridgeAuthoritati...2020-06-13T15:06:48ZRoger DingledineAssertion my_rsa_cert failed in sr_generate_our_commit when BridgeAuthoritativeDir 1Running Tor 0.2.9.9:
```
$ src/or/tor BridgeAuthoritativeDir 1
Mar 01 01:09:03.633 [notice] Tor 0.2.9.9 (git-56788a2489127072) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.1t and Zlib 1.2.8.
Mar 01 01:09:03.634 [notice] Tor ...Running Tor 0.2.9.9:
```
$ src/or/tor BridgeAuthoritativeDir 1
Mar 01 01:09:03.633 [notice] Tor 0.2.9.9 (git-56788a2489127072) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.1t and Zlib 1.2.8.
Mar 01 01:09:03.634 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Mar 01 01:09:03.634 [notice] Read configuration file "/usr/local/etc/tor/torrc".
Mar 01 01:09:03.638 [warn] ControlPort is open, but no authentication method has been configured. This means that any program on your computer can reconfigure your Tor. That's bad! You should upgrade your Tor controller as soon as possible.
Mar 01 01:09:03.639 [notice] Opening Socks listener on 127.0.0.1:9050
Mar 01 01:09:03.639 [notice] Opening Control listener on 127.0.0.1:9051
Mar 01 01:09:03.639 [notice] Opening Control listener on /home/arma/.tor/control
Mar 01 01:09:03.639 [warn] Your log may contain sensitive information - you disabled SafeLogging, and you're logging more than "notice". Don't log unless it serves an important reason. Overwrite the log afterwards.
Mar 01 01:09:03.647 [notice] Configured to measure statistics. Look for the *-stats files that will first be written to the data directory in 24 hours from now.
Mar 01 01:09:03.715 [notice] Bootstrapped 0%: Starting
Mar 01 01:09:04.220 [notice] Bootstrapped 80%: Connecting to the Tor network
Mar 01 01:09:04.220 [err] tor_assertion_failed_(): Bug: src/or/shared_random.c:908: sr_generate_our_commit: Assertion my_rsa_cert failed; aborting. (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: Assertion my_rsa_cert failed in sr_generate_our_commit at src/or/shared_random.c:908. Stack trace: (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(log_backtrace+0x42) [0x7f6fab7a1442] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(tor_assertion_failed_+0x8c) [0x7f6fab7b917c] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(sr_generate_our_commit+0x2ce) [0x7f6fab6b251e] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(sr_state_update+0x239) [0x7f6fab6b4e49] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(sr_state_init+0x8c) [0x7f6fab6b531c] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(do_main_loop+0x486) [0x7f6fab69ea16] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(tor_main+0x1c25) [0x7f6fab6a1ee5] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(main+0x19) [0x7f6fab69a179] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: /lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xf5) [0x7f6fa9c8db45] (on Tor 0.2.9.9 56788a2489127072)
Mar 01 01:09:04.221 [err] Bug: src/or/tor(+0x401c9) [0x7f6fab69a1c9] (on Tor 0.2.9.9 56788a2489127072)
Aborted
```
Probably we want to handle this weird error case more gracefully.
Pointed out by a user on #tor who seemed to be confused and enabling all sorts of config options.Tor: 0.3.1.x-finalGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/legacy/trac/-/issues/21592Make sure linked and linked_conn can't get out of sync2020-06-13T15:06:48ZteorMake sure linked and linked_conn can't get out of syncIn #21576, we encountered a situation where linked is true, but linked_conn is NULL.
This shouldn't happen, so we need to diagnose the error, and fix it.In #21576, we encountered a situation where linked is true, but linked_conn is NULL.
This shouldn't happen, so we need to diagnose the error, and fix it.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21598Log a message when a hidden service has fewer introduction points than expected2020-06-13T15:06:51ZteorLog a message when a hidden service has fewer introduction points than expectedDiagnostic for bugs like #21446.Diagnostic for bugs like #21446.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21599Make hidden service descriptor creation more consistent2020-06-13T15:06:51ZteorMake hidden service descriptor creation more consistentThis cleans up #21596: now that circuit_established is reliable, it can be used during descriptor creation as well. This prevents a regression to bugs like #21594.This cleans up #21596: now that circuit_established is reliable, it can be used during descriptor creation as well. This prevents a regression to bugs like #21594.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21622Log a message when a hidden service delays new introduction point circuits2020-06-13T15:06:55ZteorLog a message when a hidden service delays new introduction point circuitsWhen a hidden service stops making new introduction point circuits, it would be useful to log a message saying:
* how many connections it has made so far,
* how long it took it to make those connections,
* what the connection limit is,
*...When a hidden service stops making new introduction point circuits, it would be useful to log a message saying:
* how many connections it has made so far,
* how long it took it to make those connections,
* what the connection limit is,
* in what time that many connections can be made, and
* a list of the current intro point connections.
This is a follow-up to #21599.Tor: 0.3.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/21640crash on opening2020-06-13T15:06:55ZTraccrash on opening[Sun Mar 5 10:23:23 2017] Tor Software Error - The Tor software encountered an internal bug. Please report the following error message to the Tor developers at bugs.torproject.org: "microdesc_free(): Bug: microdesc_free() called, but md ...[Sun Mar 5 10:23:23 2017] Tor Software Error - The Tor software encountered an internal bug. Please report the following error message to the Tor developers at bugs.torproject.org: "microdesc_free(): Bug: microdesc_free() called, but md was still referenced 1 node(s); held_by_nodes == 1
"
**Trac**:
**Username**: kilfederTor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21641Prop274: Rotate onion keys less frequently2020-06-13T15:06:55ZNick MathewsonProp274: Rotate onion keys less frequentlyLet's implement proposal 274: This proposal will save a fair amount of bandwidth now, and even more once we have consensus diffs.Let's implement proposal 274: This proposal will save a fair amount of bandwidth now, and even more once we have consensus diffs.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21643Prop140: Extract, test, revise, and clean the diff code2020-06-13T15:06:58ZNick MathewsonProp140: Extract, test, revise, and clean the diff codeStep one of getting the consensus diff support of prop140 completed is to get the diff and patch code merged. Let's do this separately from the directory code, since that code is hairier, and may need more work.Step one of getting the consensus diff support of prop140 completed is to get the diff and patch code merged. Let's do this separately from the directory code, since that code is hairier, and may need more work.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21644Prop140: Fuzz the diff and patch code2020-06-13T15:06:58ZNick MathewsonProp140: Fuzz the diff and patch codeNow that we have fuzzing support, it would be ridiculous to start using diff/patch code without testing it through our fuzzers. So, let's do that.Now that we have fuzzing support, it would be ridiculous to start using diff/patch code without testing it through our fuzzers. So, let's do that.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21645prop140 / compression: Unified directory cache backend2020-06-13T15:06:59ZNick Mathewsonprop140 / compression: Unified directory cache backendWith prop140 and/or ahf's compression proposal, we'll need to cache more documents than previously, to include multiple consensuses, diffs, compressed consensuses, etc. We could do this in an ad-hoc way, but we'll probably be better off...With prop140 and/or ahf's compression proposal, we'll need to cache more documents than previously, to include multiple consensuses, diffs, compressed consensuses, etc. We could do this in an ad-hoc way, but we'll probably be better off with a real abstraction here.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21646prop140/compression: Refactor "directory request" code2020-06-13T15:06:59ZNick Mathewsonprop140/compression: Refactor "directory request" codeOur current notion of "what is a directory request" includes a bunch of fields that are strewn around directory connections and passed to different functions. It would be nice to instead have a "directory request" type that we created a...Our current notion of "what is a directory request" includes a bunch of fields that are strewn around directory connections and passed to different functions. It would be nice to instead have a "directory request" type that we created and passed around as appropriate. This would make it easier to test our request generation/parsing code.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21647Prop140: directory caches cache multiple past diffs or consensuses2020-06-13T15:07:00ZNick MathewsonProp140: directory caches cache multiple past diffs or consensusesTo implement prop140, we need to store history about the consensus documents. This should be fairly simple if we have #21645 to work from.To implement prop140, we need to store history about the consensus documents. This should be fairly simple if we have #21645 to work from.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21648prop140: Caches generate diffs as appropriate2020-06-13T15:07:01ZNick Mathewsonprop140: Caches generate diffs as appropriateDirectory caches should pre-generate and pre-compress diffs as appropriate. It might be best to do this in a different thread or process. It might be best to pre-generate some and create others on demand.Directory caches should pre-generate and pre-compress diffs as appropriate. It might be best to do this in a different thread or process. It might be best to pre-generate some and create others on demand.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21649prop140: Caches serve diffs on request2020-06-13T15:07:02ZNick Mathewsonprop140: Caches serve diffs on requestWhen a client asks for it, caches should serve consensus diffs.
We'll want to integrate this with the new compression logic that ahf is proposing.When a client asks for it, caches should serve consensus diffs.
We'll want to integrate this with the new compression logic that ahf is proposing.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21650prop140: Clients request diffs and handle diffs in replies2020-06-13T15:07:02ZNick Mathewsonprop140: Clients request diffs and handle diffs in repliesFor the final piece of prop140, clients should ask for consensus diffs as appropriate, and handle them if they're received.
We may need proposal extensions here:
* Should clients begin doing this immediately, or should there be a tris...For the final piece of prop140, clients should ask for consensus diffs as appropriate, and handle them if they're received.
We may need proposal extensions here:
* Should clients begin doing this immediately, or should there be a tristate where "default" means "look at the consensus"?
* Should there be an option -- maybe for testing, maybe not -- that forces clients to look for directory guards that support consensus diffs?Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21651prop140/compression: Refactor directory cache spooling code2020-06-13T15:07:03ZNick Mathewsonprop140/compression: Refactor directory cache spooling codeOur current logic for spooling things from a directory server is a bit loopy. Instead we should refactor it so that "things to be spooled" is a first-class object, with implementations depending on what we're spooling. This will make l...Our current logic for spooling things from a directory server is a bit loopy. Instead we should refactor it so that "things to be spooled" is a first-class object, with implementations depending on what we're spooling. This will make lots of our directory improvement stuff easier to implement.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21654Don't use fgets() when we might get EAGAIN2020-06-13T15:09:26ZNick MathewsonDon't use fgets() when we might get EAGAINOur tor_fgets() code, added in #20988, tries to make fgets() behave as we expect with nonblocking sockets. But really, fgets() is not such a great choice for nonblocking sockets in the first place! Let's use direct fd-level IO for thos...Our tor_fgets() code, added in #20988, tries to make fgets() behave as we expect with nonblocking sockets. But really, fgets() is not such a great choice for nonblocking sockets in the first place! Let's use direct fd-level IO for those instead.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21662prop278: Add support for LZMA2 and/or Zstandard2020-06-13T15:07:06ZAlexander Færøyahf@torproject.orgprop278: Add support for LZMA2 and/or ZstandardAdd support for the compression schemes needed to implement prop#278.
See: http://facebook.github.io/zstd/ and http://7-zip.org/sdk.html for the respective libraries.Add support for the compression schemes needed to implement prop#278.
See: http://facebook.github.io/zstd/ and http://7-zip.org/sdk.html for the respective libraries.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21663prop278: Refactor the torgzip module to support additional compression schemes2020-06-13T15:07:06ZAlexander Færøyahf@torproject.orgprop278: Refactor the torgzip module to support additional compression schemesThe current torgzip module should be refactored such that the new compression schemes needed for prop#278 can fit nicely into the code.
This is the tracking bug for this task.The current torgzip module should be refactored such that the new compression schemes needed for prop#278 can fit nicely into the code.
This is the tracking bug for this task.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21664prop278: Make the current 'torgzip' module a submodule of a new 'compression'...2020-06-13T15:07:07ZAlexander Færøyahf@torproject.orgprop278: Make the current 'torgzip' module a submodule of a new 'compression' module- Create a new 'compression' module to handle all compression schemes.
- Refactor the current 'torgzip' module into a module that handles only GZip and deflate, but adheres to the 'compression' module API.
- Create modules for the new co...- Create a new 'compression' module to handle all compression schemes.
- Refactor the current 'torgzip' module into a module that handles only GZip and deflate, but adheres to the 'compression' module API.
- Create modules for the new compression schemes: LZMA2 and Zstd.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21665Prop278: Establish an upper-bound for LZMA2 memory usage2020-06-13T15:07:07ZAlexander Færøyahf@torproject.orgProp278: Establish an upper-bound for LZMA2 memory usageOur initial analysis shows that LZMA2 can be quite a memory hog, which means we should establish some sort of upper-bound for its memory usage and how we can actually enforce it.Our initial analysis shows that LZMA2 can be quite a memory hog, which means we should establish some sort of upper-bound for its memory usage and how we can actually enforce it.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21666Prop278: Code to decide whether we want to request and/or provide CPU-intensi...2020-06-13T15:07:08ZAlexander Færøyahf@torproject.orgProp278: Code to decide whether we want to request and/or provide CPU-intensive compression methodsWe need some code in Tor to decide if or when we want to use and provide CPU-intensive compression operations. The biggest concern in Prop278 is LZMA2.We need some code in Tor to decide if or when we want to use and provide CPU-intensive compression operations. The biggest concern in Prop278 is LZMA2.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21667Prop278: Handle new headers in directory.c2020-06-13T15:07:09ZAlexander Færøyahf@torproject.orgProp278: Handle new headers in directory.cHandle the newly defined headers and their new values from Prop#278 in the directory server/client code.Handle the newly defined headers and their new values from Prop#278 in the directory server/client code.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21668Prop278: Update cached_dir_t and related types to no longer assume single com...2020-06-13T15:07:09ZAlexander Færøyahf@torproject.orgProp278: Update cached_dir_t and related types to no longer assume single compresison methodUpdate the data-types for cached_dir_t to handle multiple available compression schemes.
See also: #21651Update the data-types for cached_dir_t to handle multiple available compression schemes.
See also: #21651Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21673prop140: Handle signatures correctly2020-06-13T15:07:10ZNick Mathewsonprop140: Handle signatures correctlyFor diffs to work properly, we need to check the input document and the output document in their entirety, including their signatures. Otherwise, the diffs won't apply correctly when they change the signatures!
But for *that* to work, ...For diffs to work properly, we need to check the input document and the output document in their entirety, including their signatures. Otherwise, the diffs won't apply correctly when they change the signatures!
But for *that* to work, we need to do what we can to minimize the odds that anybody has a consensus with different signatures, or with signatures organized differently.
As an alternative, we could change the diff format so that it always replaces all the old signatures with the new ones.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21677Prop140: analyze diff performance2020-06-13T15:07:11ZNick MathewsonProp140: analyze diff performanceTor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21694Tor source tarball signed with sha12017-10-25T16:49:13ZcypherpunksTor source tarball signed with sha1$ gpg --verify -v tor-0.2.9.10.tar.gz.asc
gpg: assuming signed data in 'tor-0.2.9.10.tar.gz'
gpg: Signature made Wed 01 Mar 2017 13:09:27 GMT
gpg: using RSA key 6AFEE6D49E92B601
gpg: using subkey 6AFEE6D49E92B601 instead ...$ gpg --verify -v tor-0.2.9.10.tar.gz.asc
gpg: assuming signed data in 'tor-0.2.9.10.tar.gz'
gpg: Signature made Wed 01 Mar 2017 13:09:27 GMT
gpg: using RSA key 6AFEE6D49E92B601
gpg: using subkey 6AFEE6D49E92B601 instead of primary key FE43009C4607B1FB
gpg: using pgp trust model
gpg: Good signature from "Nick Mathewson <nickm@alum.mit.edu>" [unknown]
gpg: aka "Nick Mathewson <nickm@wangafu.net>" [unknown]
gpg: aka "Nick Mathewson <nickm@freehaven.net>" [unknown]
gpg: aka "Nick Mathewson <nickm@torproject.org>" [unknown]
gpg: WARNING: This key is not certified with a trusted signature!
gpg: There is no indication that the signature belongs to the owner.
Primary key fingerprint: 2133 BC60 0AB1 33E1 D826 D173 FE43 009C 4607 B1FB
Subkey fingerprint: 7A02 B352 1DC7 5C54 2BA0 1545 6AFE E6D4 9E92 B601
gpg: binary signature, digest algorithm SHA1, key algorithm rsa4096Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21703Warn on getinfo network-status2020-06-13T15:09:55ZNick MathewsonWarn on getinfo network-statusAccordingto control-spec, 'getinfo network-status' is deprecated. We should warn whenever it's requested, so we can finally remove it in good conscience.Accordingto control-spec, 'getinfo network-status' is deprecated. We should warn whenever it's requested, so we can finally remove it in good conscience.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21715Possible error in the Tor manual (section "NumEntryGuards NUM")2020-06-13T15:07:15ZTracPossible error in the Tor manual (section "NumEntryGuards NUM")In both, the Tor -stable Manual and the Tor -alpha Manual, there is a possible error in the section "NumEntryGuards NUM".
Literally, this is the description of the "NumEntryGuards NUM" option: "If UseEntryGuards is set to 1, we will tr...In both, the Tor -stable Manual and the Tor -alpha Manual, there is a possible error in the section "NumEntryGuards NUM".
Literally, this is the description of the "NumEntryGuards NUM" option: "If UseEntryGuards is set to 1, we will try to pick a total of NUM routers as long-term entries for our circuits. If NUM is 0, we try to learn the number from the NumEntryGuards consensus parameter, and default to 3 if the consensus parameter isn’t set. (Default: 0)".
But I think the correct description would be: "If UseEntryGuards is set to 1, we will try to pick a total of NUM routers as long-term entries for our circuits. If NUM is 0, we try to learn the number from the NumEntryGuards consensus parameter, and default to 1 if the consensus parameter isn’t set. (Default: 0)". Because Tor currently chooses a single entry guard, does not choose three.
**Trac**:
**Username**: moe-szyslak-0Tor: 0.3.1.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21729make dedicated log file perms less verbose2020-06-13T15:07:17Ztoralfmake dedicated log file perms less verboseI think, that the log file permissions are too weak in moment, this helps:
```
tfoerste@t44 ~/devel/tor $ cat /tmp/log640.patch
diff --git a/src/common/log.c b/src/common/log.c
index 5f7151b..f679336 100644
--- a/src/common/log.c
+++ b/...I think, that the log file permissions are too weak in moment, this helps:
```
tfoerste@t44 ~/devel/tor $ cat /tmp/log640.patch
diff --git a/src/common/log.c b/src/common/log.c
index 5f7151b..f679336 100644
--- a/src/common/log.c
+++ b/src/common/log.c
@@ -1086,7 +1086,7 @@ add_file_log(const log_severity_list_t *severity, const char *filename,
int open_flags = O_WRONLY|O_CREAT;
open_flags |= truncate_log ? O_TRUNC : O_APPEND;
- fd = tor_open_cloexec(filename, open_flags, 0644);
+ fd = tor_open_cloexec(filename, open_flags, 0640);
if (fd<0)
return -1;
if (tor_fd_seekend(fd)<0) {
```Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21737Speed up keccak-tiny: Use a real load64 function2020-06-13T15:07:18ZNick MathewsonSpeed up keccak-tiny: Use a real load64 functionThe current load-le64 function shows up in keccak profiles. Of course, that's silly. We can do better -- and we do, in other crypto primitives.The current load-le64 function shows up in keccak profiles. Of course, that's silly. We can do better -- and we do, in other crypto primitives.Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21744Build out glossary in torspec2020-06-13T15:07:19ZChelsea KomloBuild out glossary in torspecThere are a lot of terms defined throughout torspec, but defining common terms in the glossary will hopefully be useful.
Ideas of subsections, how to organize, terms to include, etc are welcome!There are a lot of terms defined throughout torspec, but defining common terms in the glossary will hopefully be useful.
Ideas of subsections, how to organize, terms to include, etc are welcome!Tor: 0.3.1.x-finalChelsea KomloChelsea Komlohttps://gitlab.torproject.org/legacy/trac/-/issues/21750prop224: ntor handshake implementation2020-06-13T15:07:19ZDavid Gouletdgoulet@torproject.orgprop224: ntor handshake implementationTicket created after https://trac.torproject.org/projects/tor/ticket/20657#comment:12
Initial reviews are here: https://gitlab.com/asn/tor/merge_requests/13
OK after a review from David and some comments from Nick I present the `prop22...Ticket created after https://trac.torproject.org/projects/tor/ticket/20657#comment:12
Initial reviews are here: https://gitlab.com/asn/tor/merge_requests/13
OK after a review from David and some comments from Nick I present the `prop224-ntor-v2` branch which comes with all the code review fixes, and with a full on integration test suite similar to the `./src/test/test_ntor.sh` tests for simple ntor.
It also implements the key expansion functionality as requested by David.Tor: 0.3.1.x-finalGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/legacy/trac/-/issues/21757Using Pluggable Transports on tor master is broken2020-06-13T15:07:20ZGeorg KoppenUsing Pluggable Transports on tor master is brokenWhile testing a patch for #21753 I realized that PTs in general are currently broken with tor master resulting into output like
```
CMETHOD obfs2 socks5 127.0.0.1:41925
CMETHOD obfs3 socks5 127.0.0.1:41299
CMETHOD obfs4 socks5 127.0.0.1:...While testing a patch for #21753 I realized that PTs in general are currently broken with tor master resulting into output like
```
CMETHOD obfs2 socks5 127.0.0.1:41925
CMETHOD obfs3 socks5 127.0.0.1:41299
CMETHOD obfs4 socks5 127.0.0.1:34025
CMETHOD scramblesuit socks5 127.0.0.1:35225
CMETHODS DONE'. We only support version '1'
Mar 16 13:03:05.000 [warn] Managed proxy at './TorBrowser/Tor/PluggableTransports/obfs4proxy' failed the configuration protocol and will be destroyed.
```
if started from Tor Browser.
Bisecting reveals that this got introduced with 6e78ede73f190f3aaf91623a131a20cf031aad7e which fixed #21654.Tor: 0.3.1.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/21771Cannot change bridge (with Tor 0.3.0.4-rc)2020-06-13T15:07:22ZTracCannot change bridge (with Tor 0.3.0.4-rc)Trying to replace a bridge in torrc (that had been successfully used for a long time) with another one results in many warnings "[warn] Failed to find node for hop 0 of our path. Discarding this circuit.", and failure to connect.
Only w...Trying to replace a bridge in torrc (that had been successfully used for a long time) with another one results in many warnings "[warn] Failed to find node for hop 0 of our path. Discarding this circuit.", and failure to connect.
Only way out found is to edit "state" file and remove the "Guard" line for the previous bridge (with "listed=0"), or come back to previous bridge option in torrc.
I found this while testing further after reporting what I initially thought was a bug in TorBrowser (https://trac.torproject.org/projects/tor/ticket/21768)
**Trac**:
**Username**: torvlnt33rTor: 0.3.1.x-finalGeorge KadianakisGeorge Kadianakishttps://gitlab.torproject.org/legacy/trac/-/issues/21788Very small memory leak with --verify-config2020-06-13T15:45:05ZJigsaw52Very small memory leak with --verify-configWhen tor is run with the flag --verify-config, there is a memory leak of 45 bytes. Although the leak is insignificant, not leaking any memory helps testing the configuration related code against memory leaks.
This leak is due to clean_u...When tor is run with the flag --verify-config, there is a memory leak of 45 bytes. Although the leak is insignificant, not leaking any memory helps testing the configuration related code against memory leaks.
This leak is due to clean_up_backtrace_handler not being called during tor shutdown process. I believe the fix is calling it inside tor_free_all. I will post a link to a branch with my proposed fix.
Valgrind log:
```
user@lubuntu:~/Desktop/tor$ valgrind --tool=memcheck --leak-check=full --show-leak-kinds=all /usr/local/bin/tor --verify-config
==19135== Memcheck, a memory error detector
==19135== Copyright (C) 2002-2015, and GNU GPL'd, by Julian Seward et al.
==19135== Using Valgrind-3.12.0.SVN and LibVEX; rerun with -h for copyright info
==19135== Command: /usr/local/bin/tor --verify-config
==19135==
Mar 20 00:59:33.916 [notice] Tor 0.3.1.0-alpha-dev (git-411736a13250586c) running on Linux with Libevent 2.0.21-stable, OpenSSL 1.0.2g and Zlib 1.2.8.
Mar 20 00:59:34.005 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Mar 20 00:59:34.008 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Mar 20 00:59:34.036 [notice] Read configuration file "/usr/local/etc/tor/torrc".
Configuration was valid
==19135==
==19135== HEAP SUMMARY:
==19135== in use at exit: 45 bytes in 1 blocks
==19135== total heap usage: 8,709 allocs, 8,708 frees, 460,735 bytes allocated
==19135==
==19135== 45 bytes in 1 blocks are still reachable in loss record 1 of 1
==19135== at 0x4C2CB3F: malloc (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==19135== by 0x61CBB4F: __vasprintf_chk (vasprintf_chk.c:80)
==19135== by 0x2669F3: vasprintf (stdio2.h:210)
==19135== by 0x2669F3: tor_vasprintf (compat.c:539)
==19135== by 0x2669F3: tor_asprintf (compat.c:516)
==19135== by 0x2657E1: configure_backtrace_handler (backtrace.c:232)
==19135== by 0x1556AF: tor_main (main.c:3609)
==19135== by 0x14F238: main (tor_main.c:34)
==19135==
==19135== LEAK SUMMARY:
==19135== definitely lost: 0 bytes in 0 blocks
==19135== indirectly lost: 0 bytes in 0 blocks
==19135== possibly lost: 0 bytes in 0 blocks
==19135== still reachable: 45 bytes in 1 blocks
==19135== suppressed: 0 bytes in 0 blocks
==19135==
==19135== For counts of detected and suppressed errors, rerun with: -v
==19135== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
```Tor: 0.3.1.x-finalJigsaw52Jigsaw52https://gitlab.torproject.org/legacy/trac/-/issues/21796missing evutil_secure_rng_add_bytes()2020-06-13T15:07:24Zcypherpunksmissing evutil_secure_rng_add_bytes() CC src/common/compat_libevent.o
src/common/compat_libevent.c:234:3: error: implicit declaration of function
'evutil_secure_rng_add_bytes' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
evutil_secure_rng... CC src/common/compat_libevent.o
src/common/compat_libevent.c:234:3: error: implicit declaration of function
'evutil_secure_rng_add_bytes' is invalid in C99
[-Werror,-Wimplicit-function-declaration]
evutil_secure_rng_add_bytes(buf, 32);
^
src/common/compat_libevent.c:234:3: note: did you mean
'evutil_secure_rng_get_bytes'?
/tmp/include/event2/util.h:808:6: note: 'evutil_secure_rng_get_bytes' declared
here
void evutil_secure_rng_get_bytes(void *buf, size_t n);
^
src/common/compat_libevent.c:234:3: error: this function declaration is not a
prototype [-Werror,-Wstrict-prototypes]
evutil_secure_rng_add_bytes(buf, 32);
^
2 errors generated.
Makefile:4834: recipe for target 'src/common/compat_libevent.o' failed
as of libevent git commit 6541168d7037457b8e5c51cc354f11bd94e618b6, the function evutil_secure_rng_add_bytes() is only exposed for platforms with arc4random_addrandom().Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21823tor FTBFS on clang/i3862020-06-13T15:07:30Zweasel (Peter Palfrader)tor FTBFS on clang/i386https://jenkins.torproject.org/job/tor-ci-linux-master-clang/ARCHITECTURE=i386,SUITE=sid/1905/consoleFull
```
21:22:55 mv -f $depbase.Tpo $depbase.Po
21:22:55 src/common/storagedir.c:209:18: error: implicit conversion loses integer preci...https://jenkins.torproject.org/job/tor-ci-linux-master-clang/ARCHITECTURE=i386,SUITE=sid/1905/consoleFull
```
21:22:55 mv -f $depbase.Tpo $depbase.Po
21:22:55 src/common/storagedir.c:209:18: error: implicit conversion loses integer precision: '__off64_t' (aka 'long long') to 'size_t' (aka 'unsigned int') [-Werror,-Wshorten-64-to-32]
21:22:55 *sz_out = st.st_size;
21:22:55 ~ ~~~^~~~~~~
21:22:55 1 error generated.
21:22:55 Makefile:4834: recipe for target 'src/common/storagedir.o' failed
21:22:55 make[1]: *** [src/common/storagedir.o] Error 1
```Tor: 0.3.1.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/21841Refactor: include openssl headers from fewer modules2020-06-13T15:15:07ZNick MathewsonRefactor: include openssl headers from fewer modulesWhen possible, we should make it so that no part of our code other than crypto*.c and tortls*.c actually depend on openssl APIs.When possible, we should make it so that no part of our code other than crypto*.c and tortls*.c actually depend on openssl APIs.Tor: 0.3.1.x-finalNick MathewsonNick Mathewson