Tor issueshttps://gitlab.torproject.org/tpo/core/tor/-/issues2020-06-27T13:53:37Zhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25928Single DA in sandbox vs. PDS_ALLOW_SELF flag2020-06-27T13:53:37ZTracSingle DA in sandbox vs. PDS_ALLOW_SELF flagI am running a TOR network simulation in a self-contained sandbox, and only
really need a single node to act as Directory Authority. The configuration
file looks as follows (the DA's fqdn is da.sandbox.local, and its IP is
12.34.56.78):
...I am running a TOR network simulation in a self-contained sandbox, and only
really need a single node to act as Directory Authority. The configuration
file looks as follows (the DA's fqdn is da.sandbox.local, and its IP is
12.34.56.78):
# common to all nodes:
RunAsDaemon 1
TestingTorNetwork 1
UseDefaultFallbackDirs 0
DataDirectory /var/lib/tor
PidFile /var/lib/tor/pid
Log info file /var/lib/tor/info.log
SafeLogging 0
DirAuthority orport=5000 v3ident=6542F7312EE19D39E9D389CCCB1DDD342A32E94D 12.34.56.78:7000 1B494B7382C8C2D2D13FB0B5175B0B3A14E54D69
# additionally, regular onion routers (incl. the DA):
ORPort 5000
# additionally, for the DA only:
DirPort 7000
Address da.sandbox.local
OutboundBindAddress da.sandbox.local
AuthoritativeDirectory 1
V3AuthoritativeDirectory 1
V3AuthVotingInterval 10
V3AuthVoteDelay 2
V3AuthDistDelay 2
When I start the DA, I get lots of log entries (in /var/lib/tor/info.log)
complaining about the absence of any reachable directory servers:
[info] router_pick_dirserver_generic(): No dirservers are reachable. Trying them all again.
[info] router_pick_directory_server(): No reachable router entries for dirservers. Trying them all again.
[info] directory_pick_generic_dirserver(): No router found for consensus network-status fetch; falling back to dirserver list.
While the single DA eventually appears to work properly, and publishes a
consensus file containing itself as a router entry, these warnings keep
showing up periodically in the log file on an ongoing basis.
Once the DA publishes its initial one-entry consensus, I can start further
ORs which are quickly added to the consensus, and any client nodes are
easily able to build circuits and operate correctly in my sandbox network.
In an attempt to silence the DA's dirserver reachability complaints I looked
through the TOR sources, and found that a DA's ability to pick itself as its
own directory server (in function router_pick_directory_server() in file
src/or/routerlist.c) is controlled by the PDS_ALLOW_SELF flag.
This flag was originally introduced in commit 02e7a83f9, and also appears
in subsequent commits b87a7760e, 74c2bff78, and b3a690749. The latter two
commits remove code that used to set the flag (haven't spent the time to
figure out if this would have helped my single-DA scenario, though).
Currently, there appears to be no code path that sets this flag, and also
no way to request it to be set via the command line or configuration file.
Would it make sense to assume the flag is *always* set (which would always
allow a DA to pick itself as its own DA), or rather would it be better to
provide some interface (config file entry) that allows setting the flag
explicitly (maybe only in testing mode)?
**Trac**:
**Username**: somloTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25935Allow DA address to be specified as FQDN2020-06-27T13:53:36ZTracAllow DA address to be specified as FQDNIt would be very helpful, particularly in sandbox situations, to specify
the Directory Authority by FQDN hostname instead of by IP address. This
would allow us to defer picking an actual IP address until the simulation
is started, and ev...It would be very helpful, particularly in sandbox situations, to specify
the Directory Authority by FQDN hostname instead of by IP address. This
would allow us to defer picking an actual IP address until the simulation
is started, and even to use some "in-game" DNS facility to figure out
the actual address after the simulation is launched.
Right now, specifying a FQDN for the "DirAuthority" config file entry
even *partially* works already: if the FQDN happens to start with a
digit, it is correctly resolved internally using available DNS resolver
infrastructure :)
The first attached patch makes that work in all cases (even when the
FQDN hostname does *not* begin with a digit).
The second attached patch allows FQDNs to be inserted into DA certs
created using tor-gencert, and correspondingly resolved when a client
parses the downloaded DA certificate.
I realize there is ongoing work to refactor parsing the DA config entry
(ticket legacy/trac#17224), so please consider this patch set either independently
on its own merits or as part of that larger effort. In the first case,
I'd be happy to redo and resubmit the patches based on review/feedback.
**Trac**:
**Username**: somloTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/25960Allow additional header lines in bandwidth measurements documents2020-06-27T13:53:34ZjugaAllow additional header lines in bandwidth measurements documentsAs commented by teor in: https://github.com/pastly/simple-bw-scanner/pull/112#issuecomment-385108911
> get it backported to 0.2.9 and later.
> I think Tor should only warn if:
> an incomplete line contains a "bw=" key
> an incompl...As commented by teor in: https://github.com/pastly/simple-bw-scanner/pull/112#issuecomment-385108911
> get it backported to 0.2.9 and later.
> I think Tor should only warn if:
> an incomplete line contains a "bw=" key
> an incomplete line contains a "node_id=" key
> an incomplete line occurs after the first complete lineTor: 0.3.5.x-finaljugajugahttps://gitlab.torproject.org/tpo/core/tor/-/issues/26029onion_extend_cpath2020-06-27T13:53:29ZTraconion_extend_cpathtor_bug_occurred_(): Bug: !circuitbuild.c:2772: onion_extend_cpath: Non-fatal assertion info !|| client failed. (on Tor 0.3.3.5-rc 81d71f0d41adf0d8)
Bug: Non-fatal assertion info !|| client failed in onion_extend_cpath at !circuitbuild.c...tor_bug_occurred_(): Bug: !circuitbuild.c:2772: onion_extend_cpath: Non-fatal assertion info !|| client failed. (on Tor 0.3.3.5-rc 81d71f0d41adf0d8)
Bug: Non-fatal assertion info !|| client failed in onion_extend_cpath at !circuitbuild.c:2772. (Stack trace not available) (on Tor 0.3.3.5-rc 81d71f0d41adf0d8)
tor_bug_occurred_(): Bug: !circuitbuild.c:2779: onion_extend_cpath: Non-fatal assertion info failed. (on Tor 0.3.3.5-rc 81d71f0d41adf0d8)
Bug: Non-fatal assertion info failed in onion_extend_cpath at !circuitbuild.c:2779. (Stack trace not available) (on Tor 0.3.3.5-rc 81d71f0d41adf0d8)
tor_bug_occurred_(): Bug: !circuitbuild.c:2390: onion_pick_cpath_exit: Non-fatal assertion !(exit_ei == NULL) failed. (on Tor 0.3.3.5-rc 81d71f0d41adf0d8)
Bug: Non-fatal assertion !(exit_ei == NULL) failed in onion_pick_cpath_exit at !circuitbuild.c:2390. (Stack trace not available) (on Tor 0.3.3.5-rc 81d71f0d41adf0d8)
**Trac**:
**Username**: whiteshieldTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26037DirAuths should check vote signatures before parsing2021-08-23T15:16:34ZIsis LovecruftDirAuths should check vote signatures before parsingteor pointed out that vote parsing occurs before checking the votes signature (both verifying the signature and ensuring that it comes from a known valid directory authority). dgoulet confirmed this is the case:
> See dirvote.c, functi...teor pointed out that vote parsing occurs before checking the votes signature (both verifying the signature and ensuring that it comes from a known valid directory authority). dgoulet confirmed this is the case:
> See dirvote.c, function dirvote_add_vote(). You will notice that the very first thing is parsing the whole thing with networkstatus_parse_vote_from_string(). Now, as far as I can tell, the voter signature check happens in that function. However, by the time we check it out, we've tokenized the votes and parsed _many_ parts of the vote already. (If you look for check_signature_token() in that function).
>
> And then once we are done parsing, we do have a valid signature for the vote which then make us check if we know the authority with trusteddirserver_get_by_v3_auth_digest().
The issue of anyone being able to trigger a hypothetical vulnerability in one of the parsing functions aside, it's also just simply not efficient to do all the parsing work and then chuck the results at the end of `networkstatus_parse_vote_from_string()` if the signature wasn't from a valid sig from a known authority.
This issue has been apparently been present since f4ce7f9c9b4 in tor-0.2.0.3-alpha.Tor: 0.3.5.x-finalSamdneySamdneyhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26142replace uses of U64_* and I64_* macros with their C99 stdint.h or inttypes.h ...2021-09-16T14:30:20ZTaylor Yureplace uses of U64_* and I64_* macros with their C99 stdint.h or inttypes.h equivalentsThere are many macros in compat.h that try to portably handle 64-bit integer values in `printf()` and `scanf()` calls. Now that we seem to require C99, we should use the equivalent macros from stdint.h or inttypes.h instead. For exampl...There are many macros in compat.h that try to portably handle 64-bit integer values in `printf()` and `scanf()` calls. Now that we seem to require C99, we should use the equivalent macros from stdint.h or inttypes.h instead. For example, `U64_FORMAT` is equivalent to `PRIu64` in inttypes.h.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26152Improve errors on crypto/openssl_version badness2020-06-27T13:53:23ZNick MathewsonImprove errors on crypto/openssl_version badnessWhen these tests fail, they currently do so in an unhelpful way. They should log the offending strings when the version strings don't match.
Extracted from legacy/trac#25549.When these tests fail, they currently do so in an unhelpful way. They should log the offending strings when the version strings don't match.
Extracted from legacy/trac#25549.Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26155Bandwidth file Timestamp is the latest scanner result, not the file creation ...2021-02-18T15:34:40ZteorBandwidth file Timestamp is the latest scanner result, not the file creation timeThe bandwidth file timestamp should be the last time a relay was scanned. But we say it's the file creation time, which is wrong.
in torflow, the timestamp is actually the oldest of the most recent timestamps for all scanners:
https://g...The bandwidth file timestamp should be the last time a relay was scanned. But we say it's the file creation time, which is wrong.
in torflow, the timestamp is actually the oldest of the most recent timestamps for all scanners:
https://gitweb.torproject.org/torflow.git/tree/NetworkScanners/BwAuthority/aggregate.py#n409
(This is buggy in small networks, because the unmeasured node scanner may end up in a state where is never has any nodes to scan. But we are not fixing torflow bugs.)
Here's how I suggest we fix the spec issue:
Replace the initial Timestamp with:
```
Timestamp NL
[At start, exactly once.]
The Unix Epoch time in seconds of the most recent scanner result.
If there are multiple scanners which can fail independently, implementations
SHOULD take the most recent timestamp from each scanner and use the
oldest value. This ensures all the scanners continue running.
If there are scanners that do not run continuously, they SHOULD be excluded
from the timestamp calculation,
It does not follow the KeyValue format for backwards
compatibility with version 1.0.0.
```
Add a file creation date:
```
"file_created=" DateTime NL
[Zero or one time.]
The date and time timestamp in ISO 8601 format and UTC time zone
when the file was created.
This Line has been added in version 1.1.0 of this specification.
```
Add a latest bandwidth in human-readable format:
```
"latest_bandwidth=" DateTime NL
[Zero or one time.]
The date and time timestamp in ISO 8601 format and UTC time zone
of the most recent scanner result.
This time MUST be identical to the initial Timestamp line.
This duplicate value is included to make the format easier for people
to read.
This Line has been added in version 1.1.0 of this specification.
```Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26200Bandwidth List format specification: add KeyValues counting errors in Bandwid...2021-02-18T15:34:40ZjugaBandwidth List format specification: add KeyValues counting errors in Bandwidth Linesthey could be useful to understand which relays fail to be measured and why, to solve legacy/trac#16559
So far we thought to add:
- success
- error_stream
- error_circuit
- error_misc
Edit: to solve legacy/trac#16559they could be useful to understand which relays fail to be measured and why, to solve legacy/trac#16559
So far we thought to add:
- success
- error_stream
- error_circuit
- error_misc
Edit: to solve legacy/trac#16559Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26211torspec update for GETINFO md/all2020-06-27T13:53:21Zrl1987torspec update for GETINFO md/allTor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26222Add a bandwidth-file line to votes in dir-spec.txt2020-06-27T13:53:20ZteorAdd a bandwidth-file line to votes in dir-spec.txtThis is an information line in votes, it does not end up in the consensus.
See legacy/trac#3723 for details.This is an information line in votes, it does not end up in the consensus.
See legacy/trac#3723 for details.Tor: 0.3.5.x-finaljugajugahttps://gitlab.torproject.org/tpo/core/tor/-/issues/26223Allow longer bandwidth lines in bandwidth files2020-06-27T13:53:20ZteorAllow longer bandwidth lines in bandwidth filesBandwidth files have a line limit of 510 characters (512 - newline and NUL).
We should make sure this limit is high enough for future bandwidth files.
We can do this by finding out the current length of sbws and torflow lines, multiply...Bandwidth files have a line limit of 510 characters (512 - newline and NUL).
We should make sure this limit is high enough for future bandwidth files.
We can do this by finding out the current length of sbws and torflow lines, multiplying it by 4x, then rounding up to the nearest power of two.
We can use the examples in the spec if we need to:
https://gitweb.torproject.org/torspec.git/tree/bandwidth-file-spec.txt
Maybe we should wait until sbws has added all the new fields?Tor: 0.3.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26228Clarify/determine specification for padding bytes, (formerly also PADDING cell)2020-06-27T13:53:20ZdmrClarify/determine specification for padding bytes, (formerly also PADDING cell)**EDIT: strikethrough content below is now covered in legacy/trac#26870 instead**
==== Background
I was trying to interpret the tor-spec for padding bytes, and ending up asking nickm for some clarification over IRC.
nickm suggested most...**EDIT: strikethrough content below is now covered in legacy/trac#26870 instead**
==== Background
I was trying to interpret the tor-spec for padding bytes, and ending up asking nickm for some clarification over IRC.
nickm suggested most of the cc'd for the ticket - I added atagar, too.
==== Unclear areas
Here are the points that need clarification / specification:
* spec for padding bytes does not clearly say what senders `MUST` or `SHOULD` do, [[mentioning that padding is with 0 bytes](https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt?id=f6e93d9751002d970614662c8173ff2fa5b7c193#n412|only)] or [[NUL bytes](https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt?id=f6e93d9751002d970614662c8173ff2fa5b7c193#n1480|(elsewhere))]
* spec for padding bytes does not say what receivers `MUST` or `SHOULD` do, when receiving non-zero bytes in the Cell (e.g. warn? ignore?)
* ~~spec is a bit inconsistent with `PADDING` cells ^^[1^^]^^[2^^]~~
==== Discussion: padding bytes
For the padding bytes that are not part of `PADDING` cells, nickm offered the following as a non-exhaustive set of possible forward-compatible options:
* "the [padding] bytes SHOULD be zero, and that implementations MUST ignore them"
* "The first 8 padding bytes MUST be zero; all subsequent padding bytes SHOULD be randomized. Implementations MUST ignore padding bytes"
* "All padding bytes should be randomized; implemenations MUST ignore unrecognized padding bytes"
... and mentioned that "[he doesn't] know enough of the argument in favor of randomization to have a very strong preference"
==== ~~Inconsistency: `PADDING` cell payload~~
~~(see bullet above)~~
~~These references highlight the inconsistency:~~
~~^^[1^^] `PADDING: Payload is unused.` per [[3 "Cell Packet format"](https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt?id=f6e93d9751002d970614662c8173ff2fa5b7c193#n469|sec)].~~
~~ implies 0 bytes of payload, so the rest should be padded per that section~~
~~^^[2^^] `The contents of a PADDING, VPADDING, or DROP cell SHOULD be chosen randomly, and MUST be ignored.` per [[7.2 "Link padding"](https://gitweb.torproject.org/torspec.git/tree/tor-spec.txt?id=f6e93d9751002d970614662c8173ff2fa5b7c193#n1723|sec)].~~
~~ implies the payload of a `PADDING` cell actually is the rest of the size of the cell, and that it SHOULD be chosen randomly~~
~~The `PADDING` cells were mentioned in IRC but not discussed.~~
~~I think a simple change to make the spec consistent between the two sections would be this:~~
~~`PADDING: Payload contains random data. (See Sec 7.2)`~~
~~However, given the other points here, is that correct?~~Tor: 0.3.5.x-finaldmrdmrhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26270Move dirauth module code from src/or/dirauth to src/dirauth2021-09-16T14:30:20ZAlexander Færøyahf@torproject.orgMove dirauth module code from src/or/dirauth to src/dirauthDuring the modularization discussion at the network team Seattle hackfest we decided to move the dirauth module code from src/or/dirauth to src/dirauth.During the modularization discussion at the network team Seattle hackfest we decided to move the dirauth module code from src/or/dirauth to src/dirauth.Tor: 0.3.5.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/tpo/core/tor/-/issues/26281Add .editorconfig?: it makes it easier to follow coding standards style with ...2020-06-27T13:53:18ZjugaAdd .editorconfig?: it makes it easier to follow coding standards style with many editorsSee more informatin in https://editorconfig.org/
Maybe it would be usefult to include it in the source code?.See more informatin in https://editorconfig.org/
Maybe it would be usefult to include it in the source code?.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26282Smartlist index implicitly converted to int2020-06-27T13:53:18Zrl1987Smartlist index implicitly converted to intWhen building with `DEBUG_SMARTLIST`:
```
src/common/util.c:4816:45: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
const char *s = smartlist_get(env_vars, i);
...When building with `DEBUG_SMARTLIST`:
```
src/common/util.c:4816:45: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
const char *s = smartlist_get(env_vars, i);
~~~~~~~~~~~~~ ^
src/common/util.c:4846:54: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
const char *s = smartlist_get(env_vars_sorted, i);
~~~~~~~~~~~~~ ^
2 warnings generated.
```
```
src/common/util.c:4816:45: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
const char *s = smartlist_get(env_vars, i);
~~~~~~~~~~~~~ ^
src/common/util.c:4846:54: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
const char *s = smartlist_get(env_vars_sorted, i);
~~~~~~~~~~~~~ ^
2 warnings generated.
```
```
src/or/control.c:4627:43: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
const char *arg = smartlist_get(args, i);
~~~~~~~~~~~~~ ^
1 warning generated.
```
```
src/or/geoip.c:153:57: warning: implicit conversion loses integer precision: 'intptr_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
geoip_country_t *c = smartlist_get(geoip_countries, idx);
~~~~~~~~~~~~~ ^~~
1 warning generated.
```
```
src/or/control.c:4627:43: warning: implicit conversion loses integer precision: 'size_t' (aka 'unsigned long') to 'int' [-Wshorten-64-to-32]
const char *arg = smartlist_get(args, i);
~~~~~~~~~~~~~ ^
1 warning generated.
```
```
src/or/geoip.c:153:57: warning: implicit conversion loses integer precision: 'intptr_t' (aka 'long') to 'int' [-Wshorten-64-to-32]
geoip_country_t *c = smartlist_get(geoip_countries, idx);
~~~~~~~~~~~~~ ^~~
1 warning generated.
```Tor: 0.3.5.x-finalrl1987rl1987https://gitlab.torproject.org/tpo/core/tor/-/issues/26301dir-spec: update descriptor on bandwidth changes only when uptime is less tha...2021-08-23T15:16:34Zjugadir-spec: update descriptor on bandwidth changes only when uptime is less than a dayAs commented in https://trac.torproject.org/projects/tor/ticket/24104#comment:19, after changes to the code, the spec should be updatedAs commented in https://trac.torproject.org/projects/tor/ticket/24104#comment:19, after changes to the code, the spec should be updatedTor: 0.3.5.x-finaljugajugahttps://gitlab.torproject.org/tpo/core/tor/-/issues/26311Error in `/usr/bin/tor': free(): invalid next size (normal): 0x000055ed468598d02020-06-27T13:53:15ZcypherpunksError in `/usr/bin/tor': free(): invalid next size (normal): 0x000055ed468598d0
https://lists.torproject.org/pipermail/tor-relays/2018-June/015353.html
"
My very stable Tor relay 16102E458460349EE45C0901DAA6C30094A9BBEA is now
rebooting every two days since May 31, 2018.
Running Tor 0.3.3.5-rc (git-4fe9670741478...
https://lists.torproject.org/pipermail/tor-relays/2018-June/015353.html
"
My very stable Tor relay 16102E458460349EE45C0901DAA6C30094A9BBEA is now
rebooting every two days since May 31, 2018.
Running Tor 0.3.3.5-rc (git-4fe9670741478750) on Linux
with Libevent 2.0.21-stable, OpenSSL 1.1.0f, Zlib 1.2.8
You will find a short extract of the Tor log below :
May 31 23:12:27 mkuktra systemd[1]: tor@default.service: Main process
exited, code=killed, status=7/BUS
May 31 23:12:27 mkuktra systemd[1]: tor@default.service: Unit entered
failed state.
May 31 23:12:27 mkuktra systemd[1]: tor@default.service: Failed with
result 'signal'.
May 31 23:12:28 mkuktra systemd[1]: tor@default.service: Service
hold-off time over, scheduling restart.
Jun 05 03:17:59 mkuktra tor[23419]: *** Error in `/usr/bin/tor': free():
invalid next size (normal): 0x000055ed468598d0 ***
Jun 05 03:17:59 mkuktra tor[23419]: ======= Backtrace: =========
Jun 05 03:17:59 mkuktra tor[23419]:
/lib/x86_64-linux-gnu/libc.so.6(+0x70bfb)[0x7fbb01af9bfb]
Jun 05 03:17:59 mkuktra tor[23419]:
/lib/x86_64-linux-gnu/libc.so.6(+0x76fc6)[0x7fbb01afffc6]
"
trac search found also legacy/trac#15776Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26331Make queries to help with Tor rotations2020-06-27T13:53:14ZNick MathewsonMake queries to help with Tor rotationsVarious of our rotations -- especially bug triage -- would benefit from saved queries. We should add those, either as stored queries or links from one of our wiki pages.Various of our rotations -- especially bug triage -- would benefit from saved queries. We should add those, either as stored queries or links from one of our wiki pages.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/26332Write up ticket triage process as algorithm or flowchart2020-06-27T13:53:14ZNick MathewsonWrite up ticket triage process as algorithm or flowchartTor: 0.3.5.x-finalNick MathewsonNick Mathewson