Tor issueshttps://gitlab.torproject.org/tpo/core/tor/-/issues2020-06-27T13:53:05Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26503src/lib/err/torerr.c missing STDERR_FILENO on Windows after refactor2020-06-27T13:53:05ZTaylor Yusrc/lib/err/torerr.c missing STDERR_FILENO on Windows after refactorThe Windows Jenkins builds seem to be failing on master due to the torerr.c refactor:
https://jenkins.torproject.org/view/Failed+Unstable/job/tor-ci-windows-master/1962/consoleFull
```
18:59:29 depbase=`echo src/lib/err/torerr.o | sed 's...The Windows Jenkins builds seem to be failing on master due to the torerr.c refactor:
https://jenkins.torproject.org/view/Failed+Unstable/job/tor-ci-windows-master/1962/consoleFull
```
18:59:29 depbase=`echo src/lib/err/torerr.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
18:59:29 gcc -std=gnu99 -DHAVE_CONFIG_H -I. -I../tor -I../tor/src -I../tor/src/ext -I../tor/src/ext/trunnel -I../tor/src/trunnel -I../tor/src/ext -Isrc/ext -DSHARE_DATADIR="\"/usr/share\"" -DLOCALSTATEDIR="\"/usr/var\"" -DBINDIR="\"/usr/bin\"" -I../UPSTREAM/usr/include -I../UPSTREAM/usr/include -I../UPSTREAM/usr/include -g -O2 -Wno-error=redundant-decls -U_FORTIFY_SOURCE -D_FORTIFY_SOURCE=2 -fstack-protector-all -Wstack-protector -fasynchronous-unwind-tables -Wall -fno-strict-aliasing -Waddress -Warray-bounds -Wdouble-promotion -Wextra -Winit-self -Wlogical-op -Wmissing-field-initializers -Wmissing-format-attribute -Wmissing-noreturn -Woverlength-strings -Woverride-init -Wshadow -Wsync-nand -Wtrampolines -Wunused-but-set-parameter -Wunused-but-set-variable -Wunused-local-typedefs -Wvariadic-macros -W -Wfloat-equal -Wundef -Wpointer-arith -Wstrict-prototypes -Wmissing-prototypes -Wwrite-strings -Wredundant-decls -Wchar-subscripts -Wcomment -Wformat=2 -Wwrite-strings -Wnested-externs -Wbad-function-cast -Wswitch-enum -Waggregate-return -Wpacked -Wunused -Wunused-parameter -Wold-style-definition -Wmissing-declarations -Werror -MT src/lib/err/torerr.o -MD -MP -MF $depbase.Tpo -c -o src/lib/err/torerr.o ../tor/src/lib/err/torerr.c &&\
18:59:29 mv -f $depbase.Tpo $depbase.Po
18:59:29 ../tor/src/lib/err/torerr.c:35:57: error: 'STDERR_FILENO' undeclared here (not in a function)
18:59:29 make[1]: *** [src/lib/err/torerr.o] Error 1
```Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26510Make "bloomfilter set" code first-class2021-09-16T14:30:20ZNick MathewsonMake "bloomfilter set" code first-classOur bloom filter duplicated between address_set.c and digestset.c, and it's slightly different in both cases. We should extract a common implementation type, use siphash, and name it bloomfilt_set or approx_set or somethign.Our bloom filter duplicated between address_set.c and digestset.c, and it's slightly different in both cases. We should extract a common implementation type, use siphash, and name it bloomfilt_set or approx_set or somethign.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26512Rename various modules and headers from the src/common refactoring.2020-06-27T13:53:05ZNick MathewsonRename various modules and headers from the src/common refactoring.I'll use this ticket to track suggested renamings, and do them all at once.
So far I have:
```
util_malloc.[ch] -> malloc.[ch]
torlog.[ch] -> log.[ch]
crypt_ops -> ???
tm_cvt.[ch] -> tm_to_time.[ch]
```I'll use this ticket to track suggested renamings, and do them all at once.
So far I have:
```
util_malloc.[ch] -> malloc.[ch]
torlog.[ch] -> log.[ch]
crypt_ops -> ???
tm_cvt.[ch] -> tm_to_time.[ch]
```Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26522strncat() without bounds2021-08-23T15:16:34ZTracstrncat() without boundsHi Team,
https://github.com/torproject/tor/blob/master/src/lib/err/backtrace.c#L267-L268
i.e
strncat(version, " ", sizeof(version)-1);
strncat(version, tor_version, sizeof(version)-1);
Easily used incorrectly (e.g., incorrectly com...Hi Team,
https://github.com/torproject/tor/blob/master/src/lib/err/backtrace.c#L267-L268
i.e
strncat(version, " ", sizeof(version)-1);
strncat(version, tor_version, sizeof(version)-1);
Easily used incorrectly (e.g., incorrectly computing the correct maximum size to add) such as (CWE-120).
Consider strcat_s, strlcat, snprintf, or automatically resizing strings. I feel the risk is high because the length parameter appears to be a constant, instead of computing the number of characters left.
Request team to please have a look and validate.
Regards
Dhiraj
**Trac**:
**Username**: DhirajTor: 0.3.5.x-finalrl1987rl1987https://gitlab.torproject.org/tpo/core/tor/-/issues/26524Extract network utilities into a new library2021-09-16T14:30:20ZNick MathewsonExtract network utilities into a new libraryTor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26525Rename sandbox_getaddrinfo() functions.2021-09-16T14:30:21ZNick MathewsonRename sandbox_getaddrinfo() functions.From my branch for legacy/trac#26524:
```
+// XXXX rename these. They are named as though they were sandbox-only,
+// XXXX but in fact they're the only allowed entry point to getaddrinfo.
+// XXXX They don't invoke the sandbox code; the...From my branch for legacy/trac#26524:
```
+// XXXX rename these. They are named as though they were sandbox-only,
+// XXXX but in fact they're the only allowed entry point to getaddrinfo.
+// XXXX They don't invoke the sandbox code; they only have an internal cache.
```
These functions should be called something like tor_getaddrinfo...(), or tor_getaddrinfo_cache...()Tor: 0.3.5.x-finalrl1987rl1987https://gitlab.torproject.org/tpo/core/tor/-/issues/26526Split all address.h functions that can invoke the resolver.2020-06-27T13:53:04ZNick MathewsonSplit all address.h functions that can invoke the resolver.These should have consistent names, and either have their own header, or share resolve.h. Having them in the same place as functions that just do name parsing is not good practice.These should have consistent names, and either have their own header, or share resolve.h. Having them in the same place as functions that just do name parsing is not good practice.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26527Remove ATTR_NONNULL2020-06-27T13:53:04ZNick MathewsonRemove ATTR_NONNULLWe define it to be empty everywhere, since it enables optimizations we don't want. We should just remove it.We define it to be empty everywhere, since it enables optimizations we don't want. We should just remove it.Tor: 0.3.5.x-finalrl1987rl1987https://gitlab.torproject.org/tpo/core/tor/-/issues/26532Combine ipv4.h and ipv6.h into address.h?2020-06-27T13:53:04ZNick MathewsonCombine ipv4.h and ipv6.h into address.h?Suggested during a review. I'm not sure about this; I could go either way.Suggested during a review. I'm not sure about this; I could go either way.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26533Extract sandbox module into its own library2021-09-16T14:29:02ZNick MathewsonExtract sandbox module into its own librarySee branch `sandbox_refactor` , with PR at https://github.com/torproject/tor/pull/188See branch `sandbox_refactor` , with PR at https://github.com/torproject/tor/pull/188Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26534Split file-access and filesystem-access stuff into its own library2021-09-16T14:29:02ZNick MathewsonSplit file-access and filesystem-access stuff into its own libraryOkay, this branch is `fs_refactor`.
It's based on my sandbox_refactor branch (see legacy/trac#26533) , so I made a PR at https://github.com/nmathewson/tor/pull/1 if you only want to see the changes since that branch.Okay, this branch is `fs_refactor`.
It's based on my sandbox_refactor branch (see legacy/trac#26533) , so I made a PR at https://github.com/nmathewson/tor/pull/1 if you only want to see the changes since that branch.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26538Extract various string, encoding, and formatting logic from config2021-09-16T14:29:02ZNick MathewsonExtract various string, encoding, and formatting logic from configSee branch `format_refactor`, with PR at https://github.com/torproject/tor/pull/190See branch `format_refactor`, with PR at https://github.com/torproject/tor/pull/190Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26541Fix minor mistakes in the bandwidth-file dir-spec entry2020-06-27T13:53:03ZteorFix minor mistakes in the bandwidth-file dir-spec entryatagar pointed out the following errors in the bandwidth-file dir-spec entry:
The bandwidth-file line appears:
```
[At most once for votes; does not occur in consensuses.]
```
He also suggested that we could change the name to bandwidt...atagar pointed out the following errors in the bandwidth-file dir-spec entry:
The bandwidth-file line appears:
```
[At most once for votes; does not occur in consensuses.]
```
He also suggested that we could change the name to bandwidth-file-headers or bandwidth-headers if we want. I don't mind what we call it, but it has to match legacy/trac#3723.
I also noticed that the definition of Value is wrong, it should be:
```
Value ::= ArgumentCharValue+
ArgumentCharValue ::= any printing ASCII character except NL and SP.
```
We should also add the new file_created and latest_bandwidth header keywords from bandwidth-file-spec.
Edit: file_created and latest_bandwidth were added in legacy/trac#26155 and it has been merged 3 now to master
Edit: previous line is about adding them to dir-specTor: 0.3.5.x-finaljugajugahttps://gitlab.torproject.org/tpo/core/tor/-/issues/26549Revision counter for v3 ephemeral hidden service is lost2021-09-30T13:45:55ZTracRevision counter for v3 ephemeral hidden service is lostWhen a controller is using a client to provide two or more v3 ephemeral hidden services, with the private keys managed by the controller, and there's a client session where the controller activates one of the hidden services but not the ...When a controller is using a client to provide two or more v3 ephemeral hidden services, with the private keys managed by the controller, and there's a client session where the controller activates one of the hidden services but not the others, the revision counters for the other hidden services are lost. This prevents the other services from being activated in future sessions because their descriptors are rejected by the HSDirs.
This happens because increment_descriptor_revision_counter() in hs_service.c calls update_revision_counters_in_state(), which loops over all the services currently being provided by the client, saves their counters, and removes any other counters from the state file. Thus if any hidden service is activated during a session, the revision counters of any services not activated during that session are lost.
Steps to reproduce:
* Use `ADD_ONION NEW:ED25519-V3 ...` to create two hidden services
* Save the private keys
* Shut down and restart tor
* Use `ADD_ONION ED25519-V3:<private_key_1> ...` to activate the first service
* Shut down and restart tor
* Use `SETEVENTS HS_DESC` to register for HS descriptor events
* Use `ADD_ONION ED25519-V3:<private_key_1> ...` to activate the first service
* The descriptor should be published successfully
* Use `ADD_ONION ED25519-V3:<private_key_2> ...` to activate the second service
* The controller receives `HS_DESC_FAILED` events with `REASON=UPLOAD_REJECTED`
It looks like this bug is related to legacy/trac#25552. I don't know whether the solution to that ticket will fix it.
**Trac**:
**Username**: akwizgranTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26558Remove nearly all of src/common/util.h and src/common/compat.h2021-09-16T14:29:02ZNick MathewsonRemove nearly all of src/common/util.h and src/common/compat.hI'll be honest here: When I started on this branch, its name was `freestyle_refactor`I'll be honest here: When I started on this branch, its name was `freestyle_refactor`Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26560install more optional libs for Travis CI2020-06-27T13:53:02ZTaylor Yuinstall more optional libs for Travis CIWe have optional dependencies on libseccomp and libcap, but we're not building with them in Travis CI. We should add them.We have optional dependencies on libseccomp and libcap, but we're not building with them in Travis CI. We should add them.Tor: 0.3.5.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26564Tor compilation fails when cross-compiling for macOS2021-09-16T14:29:02ZGeorg KoppenTor compilation fails when cross-compiling for macOSOur latest nightly build is failing when (cross)-compiling `tor` for macOS:
```
CC src/lib/process/env.o
src/lib/process/env.c:40:11: warning: implicit declaration of function '_NSGetEnviron' is invalid in C99 [-Wimplicit-functio...Our latest nightly build is failing when (cross)-compiling `tor` for macOS:
```
CC src/lib/process/env.o
src/lib/process/env.c:40:11: warning: implicit declaration of function '_NSGetEnviron' is invalid in C99 [-Wimplicit-function-declaration]
return *_NSGetEnviron();
^
src/lib/process/env.c:40:10: error: indirection requires pointer operand ('int' invalid)
return *_NSGetEnviron();
^~~~~~~~~~~~~~~~
1 warning and 1 error generated.
Makefile:7498: recipe for target 'src/lib/process/env.o' failed
```Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26572Remove src/common/compat.[ch and src/common/util.c: Make util.h a stub2021-09-16T14:29:02ZNick MathewsonRemove src/common/compat.[ch and src/common/util.c: Make util.h a stubSee branch `remove_util_and_compat`. PR at https://github.com/torproject/tor/pull/193See branch `remove_util_and_compat`. PR at https://github.com/torproject/tor/pull/193Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26593Split even more things out of or.h2021-09-16T14:29:03ZNick MathewsonSplit even more things out of or.hTor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26594Tor becomes unresponsive after configuration handling on Windows2020-06-27T13:53:01ZAlexander Færøyahf@torproject.orgTor becomes unresponsive after configuration handling on WindowsWhen trying to start Tor from master (as of commit `adcd1d8b9ac09f3abc11e2e3187fe363ad3df2fd`), Tor will never progress from the initial configuration handling that is happening before we bootstrap.
When starting Tor the following outpu...When trying to start Tor from master (as of commit `adcd1d8b9ac09f3abc11e2e3187fe363ad3df2fd`), Tor will never progress from the initial configuration handling that is happening before we bootstrap.
When starting Tor the following output will appear:
```
PS C:\Users\ahf\AppData\Local\Packages\TheDebianProject.DebianGNULinux_76v4gfsz19hv4\LocalState\rootfs\home\ahf\src\github.com\ahf\tor-win32\src\tor> .\src\or\tor.exe
Jul 01 20:58:33.065 [notice] Tor 0.3.5.0-alpha-dev (git-adcd1d8b9ac09f3a) running on Windows 8 with Libevent 2.1.8-stable, OpenSSL 1.0.2n, Zlib 1.2.11, Liblzma N/A, and Libzstd N/A.
Jul 01 20:58:33.068 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jul 01 20:58:33.069 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Jul 01 20:58:33.084 [notice] Configuration file "C:\Users\ahf\AppData\Roaming\tor\torrc" not present, using reasonable defaults.
Jul 01 20:58:33.086 [warn] Path for GeoIPFile (<default>) is relative and will resolve to C:\Users\ahf\AppData\Local\Packages\TheDebianProject.DebianGNULinux_76v4gfsz19hv4\LocalState\rootfs\home\ahf\src\github.com\ahf\tor-win32\src\tor\<default>. Is this what you wanted?
Jul 01 20:58:33.086 [warn] Path for GeoIPv6File (<default>) is relative and will resolve to C:\Users\ahf\AppData\Local\Packages\TheDebianProject.DebianGNULinux_76v4gfsz19hv4\LocalState\rootfs\home\ahf\src\github.com\ahf\tor-win32\src\tor\<default>. Is this what you wanted?
```
After this the process becomes unresponsive. The "Scheduler type KISTLite has been enabled." NOTICE log message is expected here, but is never shown.
### Bisecting the issue
I tried to bisect the issue and it looks like the issue began to show up after commit `696f6f15697260255146d634e1529202cc4c2b77` (the commit itself doesn't compile on Windows, but the following couple of commits are related fixes to that patch). The first commit that I can compile where this issue appears is `0b7452eeb2f2dee7acefee2d3ca2cb402a877ea1`.
### Analysis
I managed to track the issue down to a call to `strcasecmp()` in `config_lines_eq()` in `src/lib/encoding/confline.c`. I added the following debug output to get a slightly better understanding of what was going on (since gdb wasn't of much help here):
```
diff --git a/src/lib/encoding/confline.c b/src/lib/encoding/confline.c
index 7f535b321..4544465d3 100644
--- a/src/lib/encoding/confline.c
+++ b/src/lib/encoding/confline.c
@@ -14,6 +14,9 @@
#include "lib/string/util_string.h"
#include <string.h>
+#include <stdio.h>
+
+#define AHF_DEBUG(...) do { printf(__VA_ARGS__); fflush(stdout); } while (0)
/** Helper: allocate a new configuration option mapping 'key' to 'val',
* append it to *<b>lst</b>. */
@@ -232,8 +235,26 @@ int
config_lines_eq(config_line_t *a, config_line_t *b)
{
while (a && b) {
+ AHF_DEBUG("a: %p\n", a);
+ AHF_DEBUG("b: %p\n", b);
+
+ AHF_DEBUG("a->key: %p\n", a->key);
+ AHF_DEBUG("a->value: %p\n", a->value);
+ AHF_DEBUG("a->next: %p\n\n", a->next);
+
+ AHF_DEBUG("b->key: %p\n", b->key);
+ AHF_DEBUG("b->value: %p\n", b->value);
+ AHF_DEBUG("b->next: %p\n\n", b->next);
+
+ AHF_DEBUG("a->key: '%s'\n", a->key);
+ AHF_DEBUG("a->value: '%s'\n", a->value);
+
+ AHF_DEBUG("b->key: '%s'\n", b->key);
+ AHF_DEBUG("b->value: '%s'\n", b->value);
+
if (strcasecmp(a->key, b->key) || strcmp(a->value, b->value))
return 0;
a = a->next;
b = b->next;
}
```
Tor would now output the following when started:
```
$ ./src/or/tor.exe
Jul 01 21:24:47.029 [notice] Tor 0.3.5.0-alpha-dev (git-0b7452eeb2f2dee7) running on Very recent version of Windows [major=10,minor=0] with Libevent 2.1.8-stable, OpenSSL 1.0.2o, Zlib 1.2.11, Liblzma 5.2.4, and Libzstd N/A.
Jul 01 21:24:47.029 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jul 01 21:24:47.029 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Jul 01 21:24:47.039 [notice] Configuration file "C:\Users\ahf\AppData\Roaming\tor\torrc" not present, using reasonable defaults.
Jul 01 21:24:47.041 [warn] Path for GeoIPFile (<default>) is relative and will resolve to C:\msys64\home\ahf\src\github.com\ahf\tor\<default>. Is this what you wanted?
Jul 01 21:24:47.041 [warn] Path for GeoIPv6File (<default>) is relative and will resolve to C:\msys64\home\ahf\src\github.com\ahf\tor\<default>. Is this what you wanted?
a: 0000000001C6C9E0
b: 0000000001C6CE60
a->key: 0000000001C6CC50
a->value: 00000000035DAC70
a->next: 0000000000000000
b->key: 0000000001C6CDA0
b->value: 00000000035DACC0
b->next: 0000000000000000
a->key: 'TestingV3AuthInitialVotingInterval'
a->value: '1800'
b->key: 'TestingV3AuthInitialVotingInterval'
b->value: '1800'
```
And after that it would become unresponsive again.
If we try to change the call from `strcasecmp()` to `strcmp()` in `config_lines_eq()` Tor will progress and bootstrap successfully:
```
$ ./src/or/tor.exe
Jul 01 21:29:19.667 [notice] Tor 0.3.5.0-alpha-dev (git-0b7452eeb2f2dee7) running on Very recent version of Windows [major=10,minor=0] with Libevent 2.1.8-stable, OpenSSL 1.0.2o, Zlib 1.2.11, Liblzma 5.2.4, and Libzstd N/A.
Jul 01 21:29:19.667 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
Jul 01 21:29:19.667 [notice] This version is not a stable Tor release. Expect more bugs than usual.
Jul 01 21:29:19.678 [notice] Configuration file "C:\Users\ahf\AppData\Roaming\tor\torrc" not present, using reasonable defaults.
Jul 01 21:29:19.679 [warn] Path for GeoIPFile (<default>) is relative and will resolve to C:\msys64\home\ahf\src\github.com\ahf\tor\<default>. Is this what you wanted?
Jul 01 21:29:19.679 [warn] Path for GeoIPv6File (<default>) is relative and will resolve to C:\msys64\home\ahf\src\github.com\ahf\tor\<default>. Is this what you wanted?
a: 000000000139CB30
b: 000000000139C950
a->key: 000000000139C8C0
a->value: 000000000365AF30
a->next: 0000000000000000
b->key: 000000000139C7D0
b->value: 000000000365AF40
b->next: 0000000000000000
a->key: 'TestingV3AuthInitialVotingInterval'
a->value: '1800'
b->key: 'TestingV3AuthInitialVotingInterval'
b->value: '1800'
[ ... some debug output omitted here ... ]
Jul 01 21:29:19.683 [notice] Scheduler type KISTLite has been enabled.
Jul 01 21:29:19.683 [notice] Opening Socks listener on 127.0.0.1:9050
Jul 01 21:29:19.000 [notice] Bootstrapped 0%: Starting
Jul 01 21:29:20.000 [notice] Starting with guard context "default"
Jul 01 21:29:20.000 [notice] Bootstrapped 80%: Connecting to the Tor network
```
But since our configuration keys are supposed to be case insensitive this is not a fix for the problem.
### Reproducing
I have tested this in two different environments: 64-bit Tor compiled via msys2 and 32-bit Tor compiled using my own build scripts from https://github.com/ahf/tor-win32 in a Debian WSL (Windows Subsystem for Linux) container. The results are the same. All of it was done on Windows 10.Tor: 0.3.5.x-final