Tor issueshttps://gitlab.torproject.org/tpo/core/tor/-/issues2020-06-27T13:52:32Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27205add unit test for double-voting2020-06-27T13:52:32Zcypherpunksadd unit test for double-voting`protover.c` has a comment about very important code that guards against double-voting:
```
smartlist_sort_strings(expanded);
smartlist_uniq_strings(expanded); // This makes voting work. do not remove
```
But no test fails if this ...`protover.c` has a comment about very important code that guards against double-voting:
```
smartlist_sort_strings(expanded);
smartlist_uniq_strings(expanded); // This makes voting work. do not remove
```
But no test fails if this is removed.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27206rust protover_all_supported() takes forever on some inputs2020-06-27T13:52:32ZTracrust protover_all_supported() takes forever on some inputsIt calls `ProtoSet::retain()` to manually check whether each individual version is supported. Potentially checking over 4 billion of them.
**Trac**:
**Username**: cyberpunksIt calls `ProtoSet::retain()` to manually check whether each individual version is supported. Potentially checking over 4 billion of them.
**Trac**:
**Username**: cyberpunksTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27211Add chutney-git-bisect.sh to Tor's scripts directory2020-06-27T13:52:32ZteorAdd chutney-git-bisect.sh to Tor's scripts directoryI made a script in legacy/trac#27080 that we should add to master.I made a script in legacy/trac#27080 that we should add to master.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27216Fix spelling error in comment for getinfo_helper_current_time()2020-06-27T13:52:31ZNeel Chauhanneel@neelc.orgFix spelling error in comment for getinfo_helper_current_time()The comment for `getinfo_helper_current_time()` is currently:
```
/** Implementation helper for GETINFO: answers requests for information about
* the current time in both local and UTF forms. */
```
But `UTF` is wrong. It should be `U...The comment for `getinfo_helper_current_time()` is currently:
```
/** Implementation helper for GETINFO: answers requests for information about
* the current time in both local and UTF forms. */
```
But `UTF` is wrong. It should be `UTC`.Tor: 0.3.5.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/27224Call node_get_all_orports() less from node_is_a_configured_bridge()2020-06-27T13:52:31ZNick MathewsonCall node_get_all_orports() less from node_is_a_configured_bridge()According to our profile, this function accounts for a huge amount of the total calls to malloc() that we do. Although I don't think it accounts for much allocation at any given time, it is probably slowing us down.According to our profile, this function accounts for a huge amount of the total calls to malloc() that we do. Although I don't think it accounts for much allocation at any given time, it is probably slowing us down.Tor: 0.3.5.x-finalrl1987rl1987https://gitlab.torproject.org/tpo/core/tor/-/issues/27246Can we use less space for RSA onion keys on clients?2020-06-27T13:52:30ZNick MathewsonCan we use less space for RSA onion keys on clients?Our heap profiles say that we're using a HUGE amount of storage for RSA onion keys. That's pretty sad, considering that we only use them for legacy v2 hidden service stuff.
Maybe we can only parse them on-demand, when we need them?
Ma...Our heap profiles say that we're using a HUGE amount of storage for RSA onion keys. That's pretty sad, considering that we only use them for legacy v2 hidden service stuff.
Maybe we can only parse them on-demand, when we need them?
Maybe (if the openssl format is much more expensive than the raw asn1) we can store them in asn1, and only convert them to EVP_PKEY objects when we need them?Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27247Clients are using RAM for cached_dir_t2020-06-27T13:52:29ZNick MathewsonClients are using RAM for cached_dir_tAccording to the profiles, clients are storing a cached_dir_t object for the consensus they have. This is ridiculous -- they don't need that, and they already have it somewhere else.
Directory caches probably don't need one of these ei...According to the profiles, clients are storing a cached_dir_t object for the consensus they have. This is ridiculous -- they don't need that, and they already have it somewhere else.
Directory caches probably don't need one of these either -- they could mmap it instead.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27250Duplicate line '#include "lib/net/address.h' in src/test/test_address.c2020-06-27T13:52:29ZNeel Chauhanneel@neelc.orgDuplicate line '#include "lib/net/address.h' in src/test/test_address.cWill submit a PR shortly.Will submit a PR shortly.Tor: 0.3.5.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/27252Reduce the number of travis jobs2020-06-27T13:52:29ZteorReduce the number of travis jobsAfter legacy/trac#24629, we have some follow-up work.
In 0.2.9 and later:
* work out if we really need clang and gcc on Linux and macOS
In 0.3.2 and later:
* reduce the number of rust online travis jobs, to reduce travis network failuresAfter legacy/trac#24629, we have some follow-up work.
In 0.2.9 and later:
* work out if we really need clang and gcc on Linux and macOS
In 0.3.2 and later:
* reduce the number of rust online travis jobs, to reduce travis network failuresTor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27272ASan is incompatible with Rust's jemalloc on Travis2020-06-27T13:52:29ZTracASan is incompatible with Rust's jemalloc on TravisIn helping to debug https://trac.torproject.org/projects/tor/ticket/25386 I've found that many of the segfaults at runtime are attributable to Rust pulling in jemalloc by default for tests, which apparently doesn't play well with ASan wh...In helping to debug https://trac.torproject.org/projects/tor/ticket/25386 I've found that many of the segfaults at runtime are attributable to Rust pulling in jemalloc by default for tests, which apparently doesn't play well with ASan when linked in.
I've found that using code like:
```
#[global_allocator]
static ALLOCATOR: std::alloc::System = std::alloc::System;
```
is enough to solve the problem. This tells Rust that it should use the system allocator (e.g. the malloc/free symbols) instead of jemalloc. This was stabilized very recently in Rust, though, so using it may not be so trivial!
In some local testing I was able to get away with adding the above declaration to the `tor_allocate` crate for the most part, but crates like `crypto`, `external`, and `smartlist` don't already link to `tor_allocate` and needed the above declaration with a `#[cfg(test)]` as well. Once this was all added though I mostly no longer saw segfaults related to jemalloc and ASan
**Trac**:
**Username**: alexcrichtonTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27273ASan fails to link on Travis due to rustc and linker arguments2020-06-27T13:52:28ZTracASan fails to link on Travis due to rustc and linker argumentsIn helping to debug https://trac.torproject.org/projects/tor/ticket/25386 I've found that a number of the link errors are attributable to the linker invocation on Travis. The recent `link_rust.sh` script is definitely necessary I think, ...In helping to debug https://trac.torproject.org/projects/tor/ticket/25386 I've found that a number of the link errors are attributable to the linker invocation on Travis. The recent `link_rust.sh` script is definitely necessary I think, except that I believe it needs a few tweaks:
* The `-lasan` argument should be replaced with `-fsanitizer=address` I believe to ensure that the C compiler links all of its relevant libraries (as they may have different names and different numbers of libs on some platforms I think).
* Second, the Rust compiler by default passes `-nodefaultlibs` to the linker which means that the linker script also passes this along. It looks, however, like this is incompatible with `-fsanitizer=address`, causing link errors. This argument should be safe to strip out though.
The first bullet was easy enough to fix locally and the second bullet was fixable by changing `link_rust.sh` to a `bash` script and then using something like:
```
linker_args="$@"
$CCLD @RUST_LINKER_OPTIONS@ ${linker_args//-nodefaultlibs/}
```
**Trac**:
**Username**: alexcrichtonTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27274ASan on OSX Travis is incompatible with Rust's santiziers2020-06-27T13:52:28ZTracASan on OSX Travis is incompatible with Rust's santiziersIn helping to debug https://trac.torproject.org/projects/tor/ticket/25386 I've found that an unfortunate case is being hit where `cargo test` is using rustc's copy of the asan runtime instead of the system's copy, which causes problems d...In helping to debug https://trac.torproject.org/projects/tor/ticket/25386 I've found that an unfortunate case is being hit where `cargo test` is using rustc's copy of the asan runtime instead of the system's copy, which causes problems due to presumably version mismatches between them.
The error looks like https://travis-ci.com/alexcrichton/tor/jobs/141409956 and only starts to show up after https://trac.torproject.org/projects/tor/ticket/27273 and https://trac.torproject.org/projects/tor/ticket/27272 are fixed.
AFAIK the only "fix" for this is to basically just delete the sanitizer runtimes in the Rust sysroot as they're not used anyway, but I'll try to keep thinking and see if there's a better solution!
**Trac**:
**Username**: alexcrichtonTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27275Stop reporting appveyor on_success, because it's noisy2020-06-27T13:52:28ZteorStop reporting appveyor on_success, because it's noisyAppveyor currently reports:
```
on_success:
- cmd: ... success
on_failure:
- cmd: ... failure
```
which is really noisy.
Travis currently reports:
```
irc:
...
on_success: change
on_failure: change
```
which seems ok.
We ...Appveyor currently reports:
```
on_success:
- cmd: ... success
on_failure:
- cmd: ... failure
```
which is really noisy.
Travis currently reports:
```
irc:
...
on_success: change
on_failure: change
```
which seems ok.
We should make Appveyor notifications more like Travis.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27286Update recommended and required protocol versions for "LinkAuth"2020-06-27T13:52:28ZNick MathewsonUpdate recommended and required protocol versions for "LinkAuth"LinkAuth method 1 is the one where we pull the TLS master secrets out of the OpenSSL data structures and authenticate them with RSA. LinkAuth method 3 is the one where we use the RFC5705 key export mechanism and Ed25519 signatures; it i...LinkAuth method 1 is the one where we pull the TLS master secrets out of the OpenSSL data structures and authenticate them with RSA. LinkAuth method 3 is the one where we use the RFC5705 key export mechanism and Ed25519 signatures; it is not supported in 0.2.9.
Right now we list method 1 as required for clients and relays. That's a problem, since we can't reasonably support it with NSS.
We should at least say that method 1 is not required for clients, and method 3 is recommended for everybody.
Should any method be required for relays? I don't think so currently, since we don't want to kick anybody off the network.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27287make distcheck should run make check2020-06-27T13:52:28Zteormake distcheck should run make checkmake distcheck is missing check-changes, and maybe other checksmake distcheck is missing check-changes, and maybe other checksTor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27288Tor with NSS must not claim to support LinkAuth=12020-06-27T13:52:28ZNick MathewsonTor with NSS must not claim to support LinkAuth=1LinkAuth=1 is the older one that pokes inside the world of TLS master secrets. NSS sensibly doesn't let us do that, and makes us use RFC5705 like sensible people.
We shouldn't claim to support it, though.
I'm making this a separate tic...LinkAuth=1 is the older one that pokes inside the world of TLS master secrets. NSS sensibly doesn't let us do that, and makes us use RFC5705 like sensible people.
We shouldn't claim to support it, though.
I'm making this a separate ticket from the rest of NSS-TLS, though, since once we merge this, Tor clients and servers will stop working with NSS until legacy/trac#27286 is merged to update the list of required protocols.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27289Count raw bytes on the wire correctly when using NSS2020-06-27T13:52:28ZNick MathewsonCount raw bytes on the wire correctly when using NSSRight now we don't support counting the raw bytes send over TCP with NSS, but it's possible. To do this, we'd need to use PRFileDesc's layering feature, and add a layer whose only purpose is to count bytes.
This change would make our ba...Right now we don't support counting the raw bytes send over TCP with NSS, but it's possible. To do this, we'd need to use PRFileDesc's layering feature, and add a layer whose only purpose is to count bytes.
This change would make our bandwidth accounting slightly more accurate with NSS.
I think it would be safe to defer this.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27295make test-network-all should clear the test_network_log directory before running2020-06-27T13:52:27Zteormake test-network-all should clear the test_network_log directory before runningIf it doesn't, stale results in the directory can cause the build to fail.If it doesn't, stale results in the directory can cause the build to fail.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27296macOS x86_64 fails test-timers2020-06-27T13:52:27ZteormacOS x86_64 fails test-timersFrom https://travis-ci.org/torproject/tor/jobs/419913265
```
FAIL: src/test/test-timers
==========================
mean difference: 874495 usec
standard deviation: 1993793.721891 usec
Either your system is under ridiculous load, or the t...From https://travis-ci.org/torproject/tor/jobs/419913265
```
FAIL: src/test/test-timers
==========================
mean difference: 874495 usec
standard deviation: 1993793.721891 usec
Either your system is under ridiculous load, or the timer backend is broken.
FAIL src/test/test-timers (exit status: 1)
```Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27316protover.c accepts arbitrary bytes in protocol names2020-06-27T13:52:26ZTracprotover.c accepts arbitrary bytes in protocol names[dir-spec.txt](https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt) defines a protocol name as a Keyword, and strictly limits what character set is allowed in a Keyword:
```
Keyword = KeywordChar+
KeywordChar ::= 'A' ... ...[dir-spec.txt](https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt) defines a protocol name as a Keyword, and strictly limits what character set is allowed in a Keyword:
```
Keyword = KeywordChar+
KeywordChar ::= 'A' ... 'Z' | 'a' ... 'z' | '0' ... '9' | '-'
```
But `"Foo_Bar=1"`, `",,,=1"`, and arbitrary Unicode strings like `"Risqu\u00e9=1"` are accepted. Bytes that aren't even valid Unicode like `"\xc1=1"` are also fine, as long as no bytes are the null byte, `=`, or the space character.
**Trac**:
**Username**: cyberpunksTor: 0.3.5.x-final