Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T15:49:03Zhttps://gitlab.torproject.org/legacy/trac/-/issues/32718Crash: Consensus diff src/lib/memarea/memarea.c:147: memarea_chunk_free_unche...2020-06-13T15:49:03ZteorCrash: Consensus diff src/lib/memarea/memarea.c:147: memarea_chunk_free_uncheckedHere's the Tor log from OpenBSD:
```
Tor[96521]: Could not apply consensus diff because an ed command was
missing a line number.
Tor[96521]: consdiff_gen_diff: Refusing to generate consensus diff
because the generated ed diff could not b...Here's the Tor log from OpenBSD:
```
Tor[96521]: Could not apply consensus diff because an ed command was
missing a line number.
Tor[96521]: consdiff_gen_diff: Refusing to generate consensus diff
because the generated ed diff could not be tested to successfully
generate the target consensus.
Tor[96521]: tor_assertion_failed_: Bug: src/lib/memarea/memarea.c:147:
memarea_chunk_free_unchecked: Assertion sent_val == 0x90806622u failed;
aborting. (on Tor 0.4.1.6 )
Tor[96521]: Bug: Assertion sent_val == 0x90806622u failed in
memarea_chunk_free_unchecked at src/lib/memarea/memarea.c:147: . (Stack
trace not available) (on Tor 0.4.1.6 )
```
Here's the original email:
https://lists.torproject.org/pipermail/tor-relays/2019-December/017950.htmlTor: unspecifiedNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/29521Update test descriptors, and work out how to keep them updated2020-06-13T15:38:15ZteorUpdate test descriptors, and work out how to keep them updatedTor's unit tests contain a bunch of test descriptors: router descriptors, extrainfo descriptors, votes, and onion service descriptors.
But these descriptors go out of date over time, because we don't keep them updated.
For example, EX_...Tor's unit tests contain a bunch of test descriptors: router descriptors, extrainfo descriptors, votes, and onion service descriptors.
But these descriptors go out of date over time, because we don't keep them updated.
For example, EX_EI_MAXIMAL in src/test/example_extrainfo.inc is no longer maximal, because we added PaddingStatistics (and maybe other statistics).
Ideally, we need:
1. tests for the oldest supported minimal and maximal descriptors
2. tests for significant changes in the minimal and maximal descriptors
3. tests for the current version's minimal and maximal descriptors
4. a test that fails if we forget to update the test descriptors for a new feature
We could implement 1 & 2 by storing minimal and maximal descriptors from each supported Tor version. We could test 3 by generating and parsing the descriptors as part of the test, and 4 by checking for new fields in the generated descriptor, that are missing from the latest stored descriptor.
Maybe we also want a test that old versions can parse newer descriptors?
We could store an archive of supported descriptors, and test it against every supported Tor version using CI and a (Travis) cron job.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28816Call a correct connection_buf_add* function based on compress_state of dir_co...2020-06-13T15:35:32ZteorCall a correct connection_buf_add* function based on compress_state of dir_connection_tIn #21377, we discovered that it is easy to set the compression state on a connection, but add uncompressed data to that connection.
We should log a bug warning when:
* conn->compress_state is not NULL, and connection_buf_add() is calle...In #21377, we discovered that it is easy to set the compression state on a connection, but add uncompressed data to that connection.
We should log a bug warning when:
* conn->compress_state is not NULL, and connection_buf_add() is called
* conn->compress_state is NULL, and connection_buf_add_compress() is calledTor: 0.4.1.x-finalrl1987rl1987https://gitlab.torproject.org/legacy/trac/-/issues/28815Refactor similar compression buffer code in dircache.c2020-06-13T15:35:31ZteorRefactor similar compression buffer code in dircache.cAfter #21377, we will have 3 copies of similar compression buffer code in dircache.c.
Copying code is a bad idea, because:
* people make mistakes when they copy, and
* when people make changes, they sometimes miss the copies.
Instead, ...After #21377, we will have 3 copies of similar compression buffer code in dircache.c.
Copying code is a bad idea, because:
* people make mistakes when they copy, and
* when people make changes, they sometimes miss the copies.
Instead, we should write a function that correctly adds compressed or uncompressed data to the connection, based on conn, compress_method, uncompressed_body_len, lifetime, estimated_len, and the data that needs to be added to the buffer.
Perhaps the dirserv_spool_* functions could help here.
Then we can call the new function from handle_get_next_bandwidth():
https://github.com/torproject/tor/blob/b03091842bc4590e11e3ac026daae8ed6d8f7554/src/feature/dircache/dircache.c#L1463-L1468
handle_get_status_vote():
https://github.com/torproject/tor/blob/8020d6fb05d9477e77c6ca554dc1288873f6115c/src/feature/dircache/dircache.c#L1034-L1048
and handle_get_keys():
https://github.com/torproject/tor/blob/8020d6fb05d9477e77c6ca554dc1288873f6115c/src/feature/dircache/dircache.c#L1292-L1310Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/25676When a client asks for a gzip-encoded consensus, the server sends zlib2020-06-13T15:23:56ZteorWhen a client asks for a gzip-encoded consensus, the server sends zlib```
$ curl -s -O --header "Accept-Encoding: gzip" 91.121.230.208:9030/tor/status-vote/current/consensus && file consensus
Accept-Encoding: gzip
Requested: status-vote/current/consensus
consensus: zlib compressed data
```
But when the cl...```
$ curl -s -O --header "Accept-Encoding: gzip" 91.121.230.208:9030/tor/status-vote/current/consensus && file consensus
Accept-Encoding: gzip
Requested: status-vote/current/consensus
consensus: zlib compressed data
```
But when the client asks for gzip-encoded descriptors, the server sends gzip:
```
Accept-Encoding: gzip
Requested: server/authority
authority: gzip compressed data, max compression, from Unix
```
See https://trac.torproject.org/projects/tor/ticket/25667#comment:11 for a full list of requested encodings, documents, and served formats.
I think this is a minor bug with no impact, because tor clients will decompress the zlib anyway. (I don't know if tor clients ever ask for gzip without zlib.)
But I'd like someone who knows the compression code better to confirm.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/24909Update dir-spec for new compression options2020-06-13T15:20:44ZteorUpdate dir-spec for new compression optionsThe existing appendix only mentions gzip:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3777The existing appendix only mentions gzip:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3777Tor: 0.3.3.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/24368Tune zstd parameters to decrease memory usage during streaming2020-06-13T15:17:48ZteorTune zstd parameters to decrease memory usage during streamingUsing the cached-microdesc-consensus that is:
```
valid-after 2017-11-21 11:00:00
```
I get the following results:
```
$ zstd cached-microdesc-consensus
$ gzip -9 cached-microdesc-consensus
$ du -h cached-microdesc-consensus*
1.9M cache...Using the cached-microdesc-consensus that is:
```
valid-after 2017-11-21 11:00:00
```
I get the following results:
```
$ zstd cached-microdesc-consensus
$ gzip -9 cached-microdesc-consensus
$ du -h cached-microdesc-consensus*
1.9M cached-microdesc-consensus
564K cached-microdesc-consensus.gz
576K cached-microdesc-consensus.zst
```
It's only 2% larger, but I thought zstd was meant to produce smaller consensuses than gzip?
Or did I get the compression settings wrong?Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/23525Add a missing call to download_status_get_next_attempt_at()2020-06-13T15:13:53ZteorAdd a missing call to download_status_get_next_attempt_at()This makes next_attempt_at consistent over the control port and elsewhere in the code.
It also makes sure it is only accessed directly in directory.c.
This is a defect on c21cfd2 in #17750 in master.This makes next_attempt_at consistent over the control port and elsewhere in the code.
It also makes sure it is only accessed directly in directory.c.
This is a defect on c21cfd2 in #17750 in master.Tor: 0.3.2.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/22927zstd tests fail with libzstd 1.3.02020-06-13T15:11:46Zteorzstd tests fail with libzstd 1.3.0When I run the unit tests on tor master:
`
[notice] Tor 0.3.2.0-alpha-dev (git-9a1338d9df938fba) running on Darwin with Libevent 2.1.8-stable, OpenSSL 1.0.2l, Zlib 1.2.11, Liblzma 5.2.3, and Libzstd 1.3.0.
`
I see the following failures...When I run the unit tests on tor master:
`
[notice] Tor 0.3.2.0-alpha-dev (git-9a1338d9df938fba) running on Darwin with Libevent 2.1.8-stable, OpenSSL 1.0.2l, Zlib 1.2.11, Liblzma 5.2.3, and Libzstd 1.3.0.
`
I see the following failures:
```
buffer/compress/zstd: [forking]
FAIL src/test/test_buffers.c:607: assert(write_to_buf_compress(buf, compress_state, "", 0, 1) OP_EQ 0): -1 vs 0
FAIL src/test/test_buffers.c:607: assert(write_to_buf_compress(buf, compress_state, "", 0, 1) OP_EQ 0): -1 vs 0
FAIL src/test/test_buffers.c:607: assert(write_to_buf_compress(buf, compress_state, "", 0, 1) OP_EQ 0): -1 vs 0
FAIL src/test/test_buffers.c:607: assert(write_to_buf_compress(buf, compress_state, "", 0, 1) OP_EQ 0): -1 vs 0
[compress/zstd FAILED]
...
util/compress_concat/zstd:
FAIL src/test/test_util.c:2414: assert(r OP_EQ 0): -1 vs 0
[compress_concat/zstd FAILED]
```Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/13963If Modified Since delay is too long when V3AuthVotingInterval is very short2020-06-13T14:41:02ZteorIf Modified Since delay is too long when V3AuthVotingInterval is very shortClients add 3 minutes to the last consensus valid after time to create an "If Modified Since" header. This works when MIN_VOTE_INTERVAL is 5 minutes.
But when the V3AuthVotingInterval is set lower in a testing network (see #13823 and #1...Clients add 3 minutes to the last consensus valid after time to create an "If Modified Since" header. This works when MIN_VOTE_INTERVAL is 5 minutes.
But when the V3AuthVotingInterval is set lower in a testing network (see #13823 and #13718), 3 minutes can be far too long to wait for an updated consensus. (For example, MIN_VOTE_INTERVAL_TESTING is 10 seconds.)
I'd like to set the "If Modified Since" header to halfway between the valid after and fresh until times, but only when the V3AuthVotingInterval is particularly short (less than 6 minutes).
By comparison, the default of 3 minutes is 1/20 the default consensus interval of 60 minutes.Tor: 0.2.6.x-finalteorteor