Tor issueshttps://gitlab.torproject.org/tpo/core/tor/-/issues2020-06-27T13:57:07Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/21530Make ExitRelay 0 the default when there is no exit policy2020-06-27T13:57:07ZteorMake ExitRelay 0 the default when there is no exit policyWe have logged a message saying we might do this for a while.
This caused chutney bug legacy/trac#17090.We have logged a message saying we might do this for a while.
This caused chutney bug legacy/trac#17090.Tor: 0.3.5.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/22156Add Rust linting/formatting tools2020-06-27T13:56:44ZChelsea KomloAdd Rust linting/formatting toolsWe need this as another initial step to support Rust development in tor.
Work will involve adding rustfmt, Clippy, and determining rules we want/don't want.
See conversation in legacy/trac#22106We need this as another initial step to support Rust development in tor.
Work will involve adding rustfmt, Clippy, and determining rules we want/don't want.
See conversation in legacy/trac#22106Tor: 0.3.5.x-finalIsis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/core/tor/-/issues/22619SessionGroup = Reading config failed2020-06-27T13:56:26ZTracSessionGroup = Reading config failedIf i specify SessionGroup as described in manual. tor stops with error.
torrc:
SocksPort 9051 SessionGroup=1
Jun 15 16:46:24.700 [warn] Invalid SocksPort option '"SessionGroup=INT"'
Jun 15 16:46:24.700 [warn] Failed to parse/validate...If i specify SessionGroup as described in manual. tor stops with error.
torrc:
SocksPort 9051 SessionGroup=1
Jun 15 16:46:24.700 [warn] Invalid SocksPort option '"SessionGroup=INT"'
Jun 15 16:46:24.700 [warn] Failed to parse/validate config: Invalid SocksPort/SocksListenAddress configuration
Jun 15 16:46:24.701 [err] Reading config failed--see warnings above.
second try
SocksPort 9051 SessionGroup=INT
Results into:
Jun 15 16:46:35.677 [warn] Invalid SocksPort option '"SessionGroup=1"'
Jun 15 16:46:35.678 [warn] Failed to parse/validate config: Invalid SocksPort/SocksListenAddress configuration
Jun 15 16:46:35.677 [warn] Invalid SocksPort option '"SessionGroup=1"'
Jun 15 16:46:35.678 [warn] Failed to parse/validate config: Invalid SocksPort/SocksListenAddress configuration
Jun 15 16:46:35.678 [err] Reading config failed--see warnings above.
**Trac**:
**Username**: acceleraTorTor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/22747Pls document relay with restricted socket count2021-07-22T16:22:39ZTracPls document relay with restricted socket countThere is documentation in the Tor manual about how to alleviate the problem of constrained socket memory (ConstrainedSockets, ConstrainedSockSize), but not about a restricted number of sockets.
The problem of a restricted number of TCP ...There is documentation in the Tor manual about how to alleviate the problem of constrained socket memory (ConstrainedSockets, ConstrainedSockSize), but not about a restricted number of sockets.
The problem of a restricted number of TCP sockets is particularly acute in OpenVZ VPSs. A vendor may offer a great deal of bandwidth, but then restrict the practical use of it by imposing a low limit on the number of sockets in use.
So... how do I tell my relay to use no more than n TCP sockets?
-------------------------------------
```
# cat /proc/user_beancounters | grep sock
numtcpsock 3 4 3000 3000
othersockbuf 46240 108960 20571088 28942177
numothersock 42 56 3000 3000
```
**Trac**:
**Username**: tmpname0901Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/23846Option to build Tor with -fPIC (Use libtool for building shared library?)2020-06-27T13:55:18ZArturo FilastòOption to build Tor with -fPIC (Use libtool for building shared library?)It seems like it's ideal to use libtool for generating a shared library. Simone made some progress on getting this working here: https://github.com/bassosimone/mkok-onion-event/pull/2/files.It seems like it's ideal to use libtool for generating a shared library. Simone made some progress on getting this working here: https://github.com/bassosimone/mkok-onion-event/pull/2/files.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/24152refactor download_status code into its own file2021-09-16T14:31:40ZTaylor Yurefactor download_status code into its own filedirectory.c is very large and difficult to navigate. Many of the functions related to manipulating a download_status_t look like they can be in a separate file. It might be a good idea to finish legacy/trac#23354 first.directory.c is very large and difficult to navigate. Many of the functions related to manipulating a download_status_t look like they can be in a separate file. It might be a good idea to finish legacy/trac#23354 first.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/24204Improve the in-process Tor API: create owning control port2020-06-27T13:55:00ZNick MathewsonImprove the in-process Tor API: create owning control portIt would be good to have an interface something like this for controller authors to use, if they're embedding Tor:
```
#ifdef _WIN32
#define CONTROL_SOCKET SOCKET
#else
#define CONTROL_SOCKET int
#endif
/**
* Tells Tor to create a sock...It would be good to have an interface something like this for controller authors to use, if they're embedding Tor:
```
#ifdef _WIN32
#define CONTROL_SOCKET SOCKET
#else
#define CONTROL_SOCKET int
#endif
/**
* Tells Tor to create a socket for an owning controller connection to the
* Tor that will be launched. Return the socket.
*/
CONTROL_SOCKET tor_main_configuration_setup_control_socket(
tor_main_configuration_t *cfg);
#idef _WIN32
/** Windows-only; experimental. Tells Tor to create an anonymous pipe
* (whatever that's called) as an owning connection for the he controller, and
* and return that pipe. */
HANDLE tor_main_configuration_setup_control_pipe(
tor_main_configuration_t *cfg);
#endif
```
We should make sure it's portable, and that it can be implemented both with libtor_runner and with in-process Tor.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/24312Update DirCache Warning2020-06-27T13:54:56ZcypherpunksUpdate DirCache WarningWhen DirCache is disabled tor 0.3.1.x says:
```
[warn] DirCache is disabled and we are configured as a relay. This may disqualify us from becoming a guard in the future.
```
Since TorBrowser ships tor 0.3.1.x we can consider this as gr...When DirCache is disabled tor 0.3.1.x says:
```
[warn] DirCache is disabled and we are configured as a relay. This may disqualify us from becoming a guard in the future.
```
Since TorBrowser ships tor 0.3.1.x we can consider this as granted (not in the future), right?Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/24629Activate osx builds on travis, at low priority2020-06-27T13:54:41ZteorActivate osx builds on travis, at low priorityWe want to activate osx builds on Travis CI.
But they can be very slow, so we need to customise the settings.
If possible, we want osx builds to behave as follows:
* if there is a pending build, and more commits are pushed to the branch...We want to activate osx builds on Travis CI.
But they can be very slow, so we need to customise the settings.
If possible, we want osx builds to behave as follows:
* if there is a pending build, and more commits are pushed to the branch, cancel the pending build and re-queue the latest commits
* let travis show builds as "complete" if osx takes a long time, but still show osx failures eventuallyTor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/24658Split/refactor crypto.h into smaller separate modules2021-09-16T14:31:18ZIsis LovecruftSplit/refactor crypto.h into smaller separate modulesThis will make it easier to maintain, as well as easier to create new/alternate implementations of portions of the code (e.g. in Rust). `crypto.h` is already somewhat neatly partitioned into sections. nickm said that likely appropriate c...This will make it easier to maintain, as well as easier to create new/alternate implementations of portions of the code (e.g. in Rust). `crypto.h` is already somewhat neatly partitioned into sections. nickm said that likely appropriate categories for code for the new modules are
> something like: rsa, stream cipher, digest+xof, prime-field dh, openssl management, PRNG, and derived functions.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/24661accept a reasonably live consensus for guard selection2020-06-27T13:54:39ZTaylor Yuaccept a reasonably live consensus for guard selectionClients with clocks skewed far enough in the future to never get a live consensus, but still have a reasonably live one, end up downloading descriptors and then getting stuck on guard selection. This is a rather bad user experience beca...Clients with clocks skewed far enough in the future to never get a live consensus, but still have a reasonably live one, end up downloading descriptors and then getting stuck on guard selection. This is a rather bad user experience because bootstrap progress appears to get stuck at 80% or 85% even though something rather fundamental (time of day) is wrong.
It's not clear that a reasonably live consensus is dangerous to use for guard selection, so always accept a reasonably live consensus instead of a live one for guard selection.
Ticket legacy/trac#2878 covers the case of deferring descriptor downloads if the consensus isn't live, which would also improve the UX but might not be necessary if we implement the solution in this ticket.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25132Add uptime messages to control protocol GETINFO2020-06-27T13:54:14ZTracAdd uptime messages to control protocol GETINFOI am helping to maintain a fleet of relays and intended to write a Prometheus exporter (that's a numerical time-series databases and monitoring system) in Python using stem in order to aggregate statistics from them, such as `traffic/rea...I am helping to maintain a fleet of relays and intended to write a Prometheus exporter (that's a numerical time-series databases and monitoring system) in Python using stem in order to aggregate statistics from them, such as `traffic/read` and `traffic/written`, from which I can then build pretty graphs and metrics visualization/analytics. I discovered that there was no way to get the uptime in the control protocol. My first approach was to parse the heartbeat message in the logs via Logstash, but that's not good enough.
This would very useful to have for sysadmins/SRE/DevOps to monitor their relays.
I have done the implementation which adds two new commands: 'uptime' (seconds) and 'uptime_str' (in the heartbeat format). They're in separate commits.
It's pushed up to this branch: https://github.com/ageis/tor/tree/control-getinfo-uptime
I also have the accompanying update to torspec: https://github.com/ageis/torspec/tree/control-getinfo-uptime
Please review and let me know what else I need to do in order to further this patch along. I actually couldn't find much information about your development/contribution processes.
**Trac**:
**Username**: ageisp0lisTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25245Crash in assert_connection_ok when changing Exit options2020-06-27T13:54:09ZtoralfCrash in assert_connection_ok when changing Exit optionsGot this today:
```
Feb 13 22:02:49.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5113: assert_connection_ok: Assertion (conn->type == CONN_TYPE_EXIT && conn->state == EXIT_CONN_STATE_RESOLVING) || connection_is_writing(con...Got this today:
```
Feb 13 22:02:49.000 [err] tor_assertion_failed_(): Bug: src/or/connection.c:5113: assert_connection_ok: Assertion (conn->type == CONN_TYPE_EXIT && conn->state == EXIT_CONN_STATE_RESOLVING) || connection_is_writing(conn) || conn->write_blocked_on_bw || (CONN_IS_EDGE(conn) && TO_EDGE_CONN(conn)->edge_blocked_on_circ) failed; aborting. (on Tor 0.3.3.2-alpha efc105716283bbdf)
```
Before I changed the config from
ExitRelay 1
IPv6Exit 1
to
#ExitRelay 1
#IPv6Exit 1
and a little bit later to
ExitRelay 0
IPv6Exit 0
and run a kill -HUP (after every config change)Tor: 0.3.5.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/25379Determine the lowest rustc version we will compile with2020-06-27T13:54:01ZIsis LovecruftDetermine the lowest rustc version we will compile withIn our docs, we say that we currently develop with nightly, and that we might stop doing that soon. In our Makefiles, we refuse to compile if rustc is <1.14. About a week ago, some of us were pretty excited that the current stable compil...In our docs, we say that we currently develop with nightly, and that we might stop doing that soon. In our Makefiles, we refuse to compile if rustc is <1.14. About a week ago, some of us were pretty excited that the current stable compiler (1.24) [will now abort if a panic occurs at an FFI boundary](https://blog.rust-lang.org/2018/02/15/Rust-1.24.html#other-good-stuff). Also, our Makefiles currently contain a linker workaround for OSX systems for rustc <1.24 (legacy/trac#25341).
We should maaaaaybe narrow down which version(s) of Rust we're using?Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25440Broken openat syscall in Sandbox mode2020-06-27T13:53:59ZTracBroken openat syscall in Sandbox modeMy version is 0.3.3.2-alpha (git-7b1d356bdb76607d).
If relevant, I am running under Debian buster/sid amd64 KVM VPS with a 4.14.24 kernel patched with grsecurity, and AppArmor enabled.
```
Mar 06 10:14:36.024 [notice] Tor 0.3.3.2-alph...My version is 0.3.3.2-alpha (git-7b1d356bdb76607d).
If relevant, I am running under Debian buster/sid amd64 KVM VPS with a 4.14.24 kernel patched with grsecurity, and AppArmor enabled.
```
Mar 06 10:14:36.024 [notice] Tor 0.3.3.2-alpha (git-7b1d356bdb76607d) running on Linux with Libevent 2.1.8-stable, OpenSSL 1.1.0g, Zlib 1.2.8, Liblzma 5.2.2, and Libzstd 1.3.3.
Mar 06 10:14:36.025 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Mar 06 10:14:36.025 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Mar 06 10:14:36.025 [notice] Read configuration file "/etc/tor/torrc".
Mar 06 10:14:36.029 [notice] Scheduler type KIST has been enabled.
Mar 06 10:14:36.029 [notice] Opening Socks listener on 127.0.0.1:9050
Mar 06 10:14:36.029 [notice] Opening DNS listener on 127.0.0.1:5353
Mar 06 10:14:36.029 [notice] Opening Transparent pf/netfilter listener on 127.0.0.1:9040
Mar 06 10:14:36.029 [notice] Opening Control listener on 127.0.0.1:9051
============================================================ T= 1520360077
(Sandbox) Caught a bad syscall attempt (syscall openat)
tor(+0x1a57ea)[0x20b99917ea]
/lib/x86_64-linux-gnu/libpthread.so.0(open64+0x4b)[0x38f248203ab]
/lib/x86_64-linux-gnu/libpthread.so.0(open64+0x4b)[0x38f248203ab]
tor(tor_open_cloexec+0x40)[0x20b9977a00]
tor(start_writing_to_file+0x17a)[0x20b998b2ea]
tor(+0x19f3cb)[0x20b998b3cb]
tor(+0x19f518)[0x20b998b518]
tor(or_state_save+0x15b)[0x20b98aa27b]
tor(+0x5488b)[0x20b984088b]
/usr/lib/x86_64-linux-gnu/libevent-2.1.so.6(+0x229ba)[0x38f25cbe9ba]
/usr/lib/x86_64-linux-gnu/libevent-2.1.so.6(event_base_loop+0x5a7)[0x38f25cbf537]
tor(do_main_loop+0x2b4)[0x20b9841604]
tor(tor_run_main+0x1025)[0x20b9843ad5]
tor(tor_main+0x3a)[0x20b983c09a]
tor(main+0x19)[0x20b983be29]
/lib/x86_64-linux-gnu/libc.so.6(__libc_start_main+0xe7)[0x38f24272a87]
tor(_start+0x2a)[0x20b983be7a]
```
It is possible this error is either due to Tor, or it could be security hardening applied to my server. Let me know in any case... Could commit ea8e9f17f52877cc795f1792acb81d7fdaff6baf be relevant?
**Trac**:
**Username**: ageisp0lisTor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25477Stop warning users about bug #210182021-09-30T13:50:09ZGeorge KadianakisStop warning users about bug #21018While working on legacy/trac#14389 we figured out that when a Tor client connects to an onion service with authorization without the proper authorization client config (or non-existent auth config), Tor will throw out 6 warnings like thi...While working on legacy/trac#14389 we figured out that when a Tor client connects to an onion service with authorization without the proper authorization client config (or non-existent auth config), Tor will throw out 6 warnings like this:
```
[warn] Failed to parse introduction points. Either the service has published a corrupt descriptor or you have provided invalid authorization data, or (maybe!) the server is deliberately serving broken data in an attempt to crash you with bug 21018.
```
We should consider removing the alarmist warn about bug legacy/trac#21018 at some point, and maybe that point is now, since it might freak out users for no reason and there is no way for us to learn whether the attack took place.Tor: 0.3.5.x-finalrl1987rl1987https://gitlab.torproject.org/tpo/core/tor/-/issues/25505Netflow padding should inspect circuitmux queues2020-06-27T13:53:56ZMike PerryNetflow padding should inspect circuitmux queuesThe netflow padding code only tries to transmit padding if the link is otherwise idle. Dgoulet pointed out that checking the channel outbuf is not enough -- since kist that is now mostly empty. We need to inspect the circuitmux queue also.The netflow padding code only tries to transmit padding if the link is otherwise idle. Dgoulet pointed out that checking the channel outbuf is not enough -- since kist that is now mostly empty. We need to inspect the circuitmux queue also.Tor: 0.3.5.x-finalMike PerryMike Perryhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25552prop224: Onion service rev counters are useless and actually harmful for scal...2021-09-30T13:46:29ZGeorge Kadianakisprop224: Onion service rev counters are useless and actually harmful for scalabilityArmadev discovered that hsv3 revision counters are harmful to scalability since if an onion service is hosted by multiple servers (like the fb one), every server should have visibility of the revision counter if they want to publish a de...Armadev discovered that hsv3 revision counters are harmful to scalability since if an onion service is hosted by multiple servers (like the fb one), every server should have visibility of the revision counter if they want to publish a descriptor.
We should figure out whether there is an easy way around that, or whether this is actually a big problem for scalable v3s. We should also consider how this works out with onionbalance-based designs.
Rev counters are there so that HSDirs (and other actors) cannot replay old HS descriptors. However, they are not really needed since now HS descriptors are only replayable for a day (before the blinded key gets refreshed), and also HSDirs could keep a replay cache of the descriptor assigned to a blinded key.
If we decide to rip them off, the way to do it is in two painful steps:
a) Remove rev counter checking from HSDirs, and do a replay cache or something.
b) In the far future, when all HSDirs have upgraded to (a), rip out the rev counter code from onion services.Tor: 0.3.5.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/25752Detangle our included headers and reduce reliance on or.h2021-09-16T14:30:52ZIsis LovecruftDetangle our included headers and reduce reliance on or.hOver the years, our code has grown to have a lot of `#include`s which are no longer necessary, and others which are over the scope of what actually needs to be included. A prime example of this is that nearly everything in `/src/or` incl...Over the years, our code has grown to have a lot of `#include`s which are no longer necessary, and others which are over the scope of what actually needs to be included. A prime example of this is that nearly everything in `/src/or` includes `or.h` which is a huge, insanely long header file with nearly every type we've ever made. It'd be nice if we could decouple things a bit more.
I made brief foray into playing with automated tools for doing this last week. First I tried https://github.com/myint/cppclean, but it wasn't able to process any of our code without displaying a python traceback, so I moved on to https://include-what-you-use.org/. iwyu worked a bit better, but it really wanted to change system headers and remove the defined safeguards (e.g. `#ifdef HAVE_SYS_SOCKET_H #include <sys/socket.h> #endif`), which feels super scary and I'm pretty sure will result in a lot of weird breakage, particularly on systems of lower tier support status and systems developers tend to not use (e.g. win32). There's a bunch of pragmas (https://github.com/include-what-you-use/include-what-you-use/blob/master/docs/IWYUPragmas.md) we could probably use to tell it to leave alone the system headers (e.g. `find src -iname "*.[ch]" -exec sed -i -e 's/#include <*>/#include <*> \/\/ IYWU pragma: keep/' {} \;` or something).
It also wanted to do things like removing an `#include or.h` and then including from that only the system headers that were used, so maybe it's also a good idea to split up `or.h`? Like we could put the system headers and compatibility stuff at the top into a `prelude.h` or something that every module includes, and then more specific stuff in other header files. It might be nice to map out or graph which sections of code tend to need to use the same headers, in order to facilitate the organisation of splitting up `or.h`.
I have a `feature/iwyu-test` [branch](https://github.com/isislovecruft/tor/tree/feature/iwyu-test) where I committed the changes that iwyu made, plus some manual fixups that I made as I was perusing the changes (sorry, I should have kept those separate probably). I had to basically discard all changes to the ref10 ed25519 implementation, because it didn't understand why there were `#include`s mid-C-function. Also it wanted to `#include <bits/socket_type.h>` in several places, which had to be replaced with `#include <sys/socket.h>` instead, no idea why it wanted the private header in the first place (also if it did, why it didn't also `#define _SYS_SOCKET_H`). The output of `configure --disable-asciidoc && make -k CC=/home/isis/code/sources/iwyu/build/include-what-you-use 2>/tmp/iwyu.out` is ~~attached~~ [here](https://people.torproject.org/~isis/docs/2018-04-09-bug25752-iwyu-output.txt). Also I compiled iwyu against LLVM/clang 3.5.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25886Have frac_nodes_with_descriptors() take and use for_direct_connect2020-06-27T13:53:39ZNick MathewsonHave frac_nodes_with_descriptors() take and use for_direct_connectOn their review for legacy/trac#25691, teor notes (about for_direct_connect):
>We should pass for_direct_conn into this function, and use node_has_preferred_descriptor().
>For the mid and exit case:
>We won't bootstrap unless we have e...On their review for legacy/trac#25691, teor notes (about for_direct_connect):
>We should pass for_direct_conn into this function, and use node_has_preferred_descriptor().
>For the mid and exit case:
>We won't bootstrap unless we have enough actual mid and exit bandwidth, even if we have mids or exits listed as our bridges.
>
>For the guard case:
>The guard case is unchanged for non-bridge clients.
>
>The bridge client case could be tricky, because:
>
> 1. compute_frac_paths_available() only checks guard-flagged nodes, not bridges
> 2. even if it did check bridges, they ~~don't have bandwidths~~ only have self-measured bandwidths
> 3. even if we used a weight of 1 for each bridge, we don't require 65% of bridges to be up to bootstrap
>
>To workaround this issue, I suggest we make f_guard = 1.0 in compute_frac_paths_available() if we are using bridges, and have at least one bridge with ~~the preferred~~ a full descriptor.
Edit: bridge clients always use full descriptors for bridges
Edit: bridges have self-measured bandwidthsTor: 0.3.5.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.org