Tor Specifications issueshttps://gitlab.torproject.org/tpo/core/torspec/-/issues2023-11-28T18:47:44Zhttps://gitlab.torproject.org/tpo/core/torspec/-/issues/80Y2038 problem in NETINFO cell2023-11-28T18:47:44ZNick MathewsonY2038 problem in NETINFO cellThe NETINFO cell includes a 4 byte timestamp counted in seconds; we should fix that to be an 8 byte timestamp before it is too late.
The easy way to do this would be to add a new link handshake version, and change the NETINFO cell in th...The NETINFO cell includes a 4 byte timestamp counted in seconds; we should fix that to be an 8 byte timestamp before it is too late.
The easy way to do this would be to add a new link handshake version, and change the NETINFO cell in that version to have an 8 byte timestamp. (But if we do this, Arti won't benefit from it until it also implements the connection padding required in link protocol 5 and later: arti#62)
The harder kludgier way to do that would be to optionally insert the high bytes of the timestamp cell at the end of the end of the NETINFO cell, after the MYADDR stuff.
A bad kludge (IMO) would to specify that parties should interpret the timestamp as having whatever high bytes would make it closest to their current time.
Implementation breakdown, if prop338 is accepted
* In tor:
* [ ] Add version 6 to accepted versions list
* [ ] Add support for netinfo cell with 8 byte time.
* [ ] Change format of netinfo cell depending on protocol version
* [ ] Unit tests for all of the above
* [ ] Integration testing with old/new clients, old/new relays
* In arti:
* [ ] Add version 6 to accepted versions list (dup)
* [ ] Add support for netinfo cell with 8 byte time. (dup)
* [ ] Change format of netinfo cell depending on protocol version (dup)
* [ ] Unit tests for all of the above (dup)
* [ ] Integration testing with old/new clients, old/new relays (dup)
* [ ] Add support for parsing channel cells differently depending on link protocol versionNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/torspec/-/issues/53find all instances of SHA-1 in our design and implementation and kill them wi...2023-11-06T13:53:08ZIsis Lovecruftfind all instances of SHA-1 in our design and implementation and kill them with fireThis is a parent ticket for finding every use of SHA-1 in our specs/design and code, detailing it, and coming up with a plan to replace it.
From [the Seattle notes](https://trac.torproject.org/projects/tor/wiki/org/meetings/2018NetworkT...This is a parent ticket for finding every use of SHA-1 in our specs/design and code, detailing it, and coming up with a plan to replace it.
From [the Seattle notes](https://trac.torproject.org/projects/tor/wiki/org/meetings/2018NetworkTeamHackfestSeattle/OldCrypto), we use truncated SHA-1 in v2 onion services and `relay_crypt_one_payload()`, and we use full width SHA-1 for relay fingerprints and, again, v2 onion services. Nick has also written [a draft document](https://gitweb.torproject.org/torspec.git/tree/proposals/ideas/xxx-what-uses-sha1.txt) detailing where we use SHA-1, however it is presently outdated and incorrect in some places.https://gitlab.torproject.org/tpo/core/torspec/-/issues/192Inconsistent representation and specification of onion key types2023-03-08T16:10:50ZIan Jacksoniwj@torproject.orgInconsistent representation and specification of onion key typesOnion keys (the intermediate-lifetime circuit keys, eg `KP_ntor`) have a type; in the past, and perhaps in the future, there were multiple algorithms supported.
These types are represented, at least, in the following ways:
* An integer...Onion keys (the intermediate-lifetime circuit keys, eg `KP_ntor`) have a type; in the past, and perhaps in the future, there were multiple algorithms supported.
These types are represented, at least, in the following ways:
* An integer field `ONION_KEY_TYPE` in an INTRODUCE1/2 message (rend-spec-v3 \[PROCESS_INTRO2]
* A string value in a hidden service descriptor inner document (rend-spec-v3 2.5.2.2 `onion-key` keyword)
* The difference between the `onion-key` and `ntor-onion-key` keywords in a router descriptor (tor-spec 2.1.1)
These things should be cross-referenced. There should be one table of key types, with consistent naming both for the types themselves, and for the enum of types of which the individual types are variants.
Also the router descriptor format is needlessly different, and worse, and should probably be fixed if we were to get the chance to overhaul it.https://gitlab.torproject.org/tpo/core/torspec/-/issues/123Annotations should be documented in dir-spec.txt2022-07-18T17:54:03Zrl1987Annotations should be documented in dir-spec.txtTor codebase supports annotating Tor directory documents with information that is useful to Tor implementation. However, in torspec there is no description about rationale and technical details of this feature. There should be.Tor codebase supports annotating Tor directory documents with information that is useful to Tor implementation. However, in torspec there is no description about rationale and technical details of this feature. There should be.https://gitlab.torproject.org/tpo/core/torspec/-/issues/144reflect that RFC3879 cites the proposed RFC3513 is now outdated by the Draft ...2022-07-18T17:53:10Zsaibatoreflect that RFC3879 cites the proposed RFC3513 is now outdated by the Draft Standard RFC4291The source Tor code `src/lib/net/address.c `still follows an old outdated internet guideline
and might block otherwise now routebable addresses.
Since we had a discussion in Bitcoin core dev
https://github.com/bitcoin/bitcoin/pull/1998...The source Tor code `src/lib/net/address.c `still follows an old outdated internet guideline
and might block otherwise now routebable addresses.
Since we had a discussion in Bitcoin core dev
https://github.com/bitcoin/bitcoin/pull/19985
So id like to point this out here and ask about Tor's view on this?
For reference'
[RFC4291](https://www.rfc-editor.org/info/rfc4291)
A fix could be:
```
diff --git a/src/lib/net/address.c b/src/lib/net/address.c
index ea6c29db9f..6600b0db06 100644
--- a/src/lib/net/address.c
+++ b/src/lib/net/address.c
@@ -241,9 +241,6 @@ tor_addr_make_null(tor_addr_t *a, sa_family_t family)
/** Return true iff <b>ip</b> is an IP reserved to localhost or local networks.
*
* If <b>ip</b> is in RFC1918 or RFC4193 or RFC4291, we will return true.
- * (fec0::/10, deprecated by RFC3879, is also treated as internal for now
- * and will return true.)
- *
* If <b>ip</b> is 0.0.0.0 or 100.64.0.0/10 (RFC6598), we will act as:
* - Internal if <b>for_listening</b> is 0, as these addresses are not
* routable on the internet and we won't be publicly accessible to clients.
@@ -287,8 +284,7 @@ tor_addr_is_internal_(const tor_addr_t *addr, int for_listening,
return 0;
if (((iph6[0] & 0xfe000000) == 0xfc000000) || /* fc00/7 - RFC4193 */
- ((iph6[0] & 0xffc00000) == 0xfe800000) || /* fe80/10 - RFC4291 */
- ((iph6[0] & 0xffc00000) == 0xfec00000)) /* fec0/10 D- RFC3879 */
+ ((iph6[0] & 0xffc00000) == 0xfe800000)) /* fe80/10 - RFC4291 */
return 1;
if (!iph6[0] && !iph6[1] && !iph6[2] &&
```
I also saw a hint to this in https://gitlab.torproject.org/tpo/core/tor/-/issues/7971 so
maybe this is already discussed and was decided to leave it as is?
Since some time has now gone by and that memo referenced is outdated also and
rfc4291 is now a Draft Standard, i wonder what the actual view of Tor on this is?https://gitlab.torproject.org/tpo/core/torspec/-/issues/84make (retroactive) proposal for DoS subsystem2022-03-17T19:41:28ZRoger Dingledinemake (retroactive) proposal for DoS subsystemIn legacy/trac#24902, dgoulet speaks of a ddos-design.txt document.
But there is no actual proposal for the overall DoS subsystem.
If we have the document around, and we just never published it, this is a great chance to notice, clean ...In legacy/trac#24902, dgoulet speaks of a ddos-design.txt document.
But there is no actual proposal for the overall DoS subsystem.
If we have the document around, and we just never published it, this is a great chance to notice, clean it up a bit, and call it proposal three-hundred-and-something. (And then maybe turn some of it into one of the spec files if that makes sense, but, one step at a time here. :)
Motivated by this month's tor-dev thread where all we have to show for the DoS subsystem design is a trac ticket number and a changelog entry.https://gitlab.torproject.org/tpo/core/torspec/-/issues/27Tor control spec doesn't properly specify reply format2022-02-21T19:12:26ZcypherpunksTor control spec doesn't properly specify reply formatThe control spec does not sufficiently specify how to generically parse multi line replies from the controller. The intent seems to be that multi line response data is terminated by a '.' line.
However, this is not specified in the con...The control spec does not sufficiently specify how to generically parse multi line replies from the controller. The intent seems to be that multi line response data is terminated by a '.' line.
However, this is not specified in the control spec section 2.3 and the reply description there is insufficient to properly recognize multi-line reply packets leading to bugs like:
https://trac.torproject.org/projects/tor/ticket/16990https://gitlab.torproject.org/tpo/core/torspec/-/issues/9"pr" lines in consensus can have trailing whitespace2022-02-21T19:12:26Zirl"pr" lines in consensus can have trailing whitespacedir-spec specifies keyword lines as:
```
KeywordLine ::= Keyword NL | Keyword WS ArgumentChar+ NL
```
However, observed in the wild:
```
pr
```
There is trailing whitespace on line 1840 of the [[19:00:00 consensus](https://collector...dir-spec specifies keyword lines as:
```
KeywordLine ::= Keyword NL | Keyword WS ArgumentChar+ NL
```
However, observed in the wild:
```
pr
```
There is trailing whitespace on line 1840 of the [[19:00:00 consensus](https://collector.torproject.org/recent/relay-descriptors/consensuses/2019-04-09-19-00-00-consensus|2019-04-09)]. It is at line 1840.
As the directory authorities all seem to agree that this trailing whitespace should be there we don't have an issue, but it's against the spec and has likely happened by accident.
If we accidentally remove the trailing whitespace, we don't have a consensus anymore.
Options for fixing this are:
* require trailing whitespace for pr lines with no arguments
* make a new consensus method that doesn't have trailing whitespace
Either way, this needs a spec change before we write any code.https://gitlab.torproject.org/tpo/core/torspec/-/issues/13List all valid DirPort urls2022-02-21T19:12:26ZDamian JohnsonList all valid DirPort urlsHi network team. Every so often folks stumble on DirPort resources Stem doesn't recognize...
https://trac.torproject.org/projects/tor/ticket/30930
https://trac.torproject.org/projects/tor/ticket/31187
Unfortunately section 'B' is the c...Hi network team. Every so often folks stumble on DirPort resources Stem doesn't recognize...
https://trac.torproject.org/projects/tor/ticket/30930
https://trac.torproject.org/projects/tor/ticket/31187
Unfortunately section 'B' is the closest enumeration we have, but even it doesn't include everything...
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3888
Could we spec a listing of all urls the DirPort supports (similar to the GETINFO section in the control-spec)?
Thanks!https://gitlab.torproject.org/tpo/core/torspec/-/issues/54Unify bandwidth related terms in dir-spec and Tor code.2022-02-21T19:12:26ZjugaUnify bandwidth related terms in dir-spec and Tor code.Unifying these terms would make following what these bandwidth do in the code and the spec less confusing.
For instance:
- what is called `bandwidthrate` [0] in the code when building the descriptor, it is called `bandwidth-avg` in `dir-...Unifying these terms would make following what these bandwidth do in the code and the spec less confusing.
For instance:
- what is called `bandwidthrate` [0] in the code when building the descriptor, it is called `bandwidth-avg` in `dir-spec.txt` [1].
- what is called `bandwidthcapacity` [2] in the code, it is called `bandwidth-observed` in `dir-spec.txt`[1]
I don't know if it is prefered to modify terms in the spec or in the code.
It'd be also helpful to add formulae or more detailed descriptions on how the different bandwidth terms are generated in dir-spec.txt
[0] https://gitweb.torproject.org/tor.git/tree/src/or/router.c#n2370
[1] https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n427
[2] https://gitweb.torproject.org/tor.git/tree/src/or/router.c#n2376https://gitlab.torproject.org/tpo/core/torspec/-/issues/14tor-spec: We should specify how clients pick between different handshake types2022-02-21T19:12:26ZAlexander Færøyahf@torproject.orgtor-spec: We should specify how clients pick between different handshake typesAs far as I can tell we do currently not specify how a client should pick between TAP or NTor handshakes in tor-spec.txt.
We do specify under TAP that the handshake should not be used because it is slow and weak, but we do not specify a...As far as I can tell we do currently not specify how a client should pick between TAP or NTor handshakes in tor-spec.txt.
We do specify under TAP that the handshake should not be used because it is slow and weak, but we do not specify anywhere how a client should pick among the set of handshakes.
This is currently not a big issue because we only have two handshake types and Ntor is much preferred over TAP, but in a future where we might have different more fancy handshakes (PQ?) then it might be good to specify the order of which clients should prefer a given handshake over another.https://gitlab.torproject.org/tpo/core/torspec/-/issues/19Clarify how ed25519 extended private keys are used in rend-spec-v3.txt2022-02-21T19:12:26ZGeorge KadianakisClarify how ed25519 extended private keys are used in rend-spec-v3.txtIn legacy/trac#24342 we started specifying how extended private keys are used in rend-spec-v3.txt but we didn't do it in a complete manner (see comment:9:ticket:24342).
We should improve the state of the spec on this regard.In legacy/trac#24342 we started specifying how extended private keys are used in rend-spec-v3.txt but we didn't do it in a complete manner (see comment:9:ticket:24342).
We should improve the state of the spec on this regard.https://gitlab.torproject.org/tpo/core/torspec/-/issues/20Clarify the bandwidth part of dir-spec2022-02-21T19:12:25ZteorClarify the bandwidth part of dir-specPeople keep asking about the precise meaning of relay bandwidths. We should make the spec clearer:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n424
In particular:
* there is a separate limit on inbound and outbound traf...People keep asking about the precise meaning of relay bandwidths. We should make the spec clearer:
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n424
In particular:
* there is a separate limit on inbound and outbound traffic
* traffic includes origin circuits and BEGINDIR requests
* let's check if traffic includes DirPort, I think it would have to
There may also be more feedback in legacy/trac#25854.
I'm tagging this fast-fix, because I can fix it fast, and it will save me time when I next explain it.https://gitlab.torproject.org/tpo/core/torspec/-/issues/17document control protocol router status format surprises when using microdesc...2022-02-21T19:12:25Zteordocument control protocol router status format surprises when using microdescriptorsIn Tor 0.3.0 (or earlier?), we made clients use microdescriptors by default. This changes the format of NS_CONTROL_PORT routerstatus lines, which breaks pytorctl, and therefore the bandwidth authorities.
Apparently, this issue can be re...In Tor 0.3.0 (or earlier?), we made clients use microdescriptors by default. This changes the format of NS_CONTROL_PORT routerstatus lines, which breaks pytorctl, and therefore the bandwidth authorities.
Apparently, this issue can be resolved by setting UseMicrodescriptors 0 or using 0.2.9.
edit: Let's update control-spec.txt to document the current (surprising) behavior and track further behavior improvements in other tickets.https://gitlab.torproject.org/tpo/core/torspec/-/issues/18Recommend that tools ignore HSDir and Guard flags on bridges2022-02-21T19:12:25ZteorRecommend that tools ignore HSDir and Guard flags on bridgesAnd are there any other flags it should avoid?
Examples:
https://atlas.torproject.org/#details/195F974256C4751CB72316187826BAEF0F038231
https://onionoo.torproject.org/details?limit=4&type=bridgeAnd are there any other flags it should avoid?
Examples:
https://atlas.torproject.org/#details/195F974256C4751CB72316187826BAEF0F038231
https://onionoo.torproject.org/details?limit=4&type=bridgehttps://gitlab.torproject.org/tpo/core/torspec/-/issues/23Describe consensus digest calculation2022-02-21T19:12:25ZDamian JohnsonDescribe consensus digest calculationHi lovely network team folks. No doubt I'm being blind but I'm having difficulty figuring out how to calculate network status document digests.
During the voting period (minutes 55-60 of the hour) I fetched the detached signatures and u...Hi lovely network team folks. No doubt I'm being blind but I'm having difficulty figuring out how to calculate network status document digests.
During the voting period (minutes 55-60 of the hour) I fetched the detached signatures and upcoming consensus. The detached signatures cite the digest...
```
% curl http://128.31.0.39:9131/tor/status-vote/next/consensus-signatures > sigs
% curl http://128.31.0.39:9131/tor/status-vote/next/consensus > next_consensus
% grep consensus-digest sigs
consensus-digest 296BA01987256A1C8EFB20E17667152DCFA50755
```
But in trying hex encoded sha1s of various ranges of the consensus I'm having difficulty getting a value that matches that. No doubt I'm missing something but the spec is unhelpfully vague saying simply 'this is the digest' without citing a section describing how it's calculated...
https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n3309
It's probably buried in there somewhere but I've skimmed through the spec a few times and it's not jumping out at me. Mind clarifying in the spec how to calculate this?
Thanks!https://gitlab.torproject.org/tpo/core/torspec/-/issues/24PT_LOG and PT_STATUS event fields unspecifed2022-02-21T19:12:25ZDamian JohnsonPT_LOG and PT_STATUS event fields unspecifedRecently Tor added PT_LOG and PT_STATUS events to the spec...
https://gitweb.torproject.org/torspec.git/commit/?id=3028cf1
https://gitweb.torproject.org/torspec.git/commit/?id=b38257e
Unfortunately the 'pt-spec.txt section 3.3.5' secti...Recently Tor added PT_LOG and PT_STATUS events to the spec...
https://gitweb.torproject.org/torspec.git/commit/?id=3028cf1
https://gitweb.torproject.org/torspec.git/commit/?id=b38257e
Unfortunately the 'pt-spec.txt section 3.3.5' section they mention does not exist, and in looking around I can't find anything that describes what these event fields are defined as ('PT=' 'TYPE=', 'CONNECT=', etc).
I started to write a stem parser for these but can't continue until this is done (I can't parse events without knowing what fields they include).
David is aware of this and plans to has kindly offered to add the missing info...
```
22:24 <+atagar> dgoulet: Your control-spec addition to descript PT_LOG and PT_STATUS
cite a pt-spec section 3.3.4 which does not exist.
22:24 <+atagar> s/descript/describe
22:29 <+atagar> dgoulet: Huh. I'm not spotting anything that lists the keyword
arguments ('PT=' and 'SEVERITY=') so guess the sections simply
missing from the spec. I need that for stem support so please
give me a nudge when the event spec's done. :)
22:59 <+dgoulet> atagar: oh hmmm I'll fix that sorry
23:17 <+atagar> Thanks! Much appreciated. :)
```https://gitlab.torproject.org/tpo/core/torspec/-/issues/8QuotedString and CString in control-spec.txt technically require escaping asc...2022-02-21T19:12:25ZDavid Fifielddcf@torproject.orgQuotedString and CString in control-spec.txt technically require escaping ascii 32 (space)control-spec.txt 2.1 [says](https://gitweb.torproject.org/torspec.git/tree/control-spec.txt?id=795420240305a6d67c0f4322993a65da4c7b6f2f#n110):
> === 2.1. Description format ===
> We use the following nonterminals from RFC 2822: `atom`, `...control-spec.txt 2.1 [says](https://gitweb.torproject.org/torspec.git/tree/control-spec.txt?id=795420240305a6d67c0f4322993a65da4c7b6f2f#n110):
> === 2.1. Description format ===
> We use the following nonterminals from RFC 2822: `atom`, `qcontent`
> ...
> `QuotedString = DQUOTE *qcontent DQUOTE`
> ...
> === 2.1.1. Notes on an escaping bug ===
> `CString = DQUOTE *qcontent DQUOTE`
RFC 2822 [defines](https://tools.ietf.org/html/rfc2822#section-3.2.5) `qcontent` thus:
```
qtext = NO-WS-CTL / ; Non white space controls
%d33 / ; The rest of the US-ASCII
%d35-91 / ; characters not including "\"
%d93-126 ; or the quote character
qcontent = qtext / quoted-pair
```
where `NO-WS-CTL` [expands to](https://tools.ietf.org/html/rfc2822#section-3.2.1)
```
NO-WS-CTL = %d1-8 / ; US-ASCII control characters
%d11 / ; that do not include the
%d12 / ; carriage return, line feed,
%d14-31 / ; and white space characters
%d127
```
In short, `qcontent` does not include the space character (ascii 32), and so according to a strict reading of the spec, anything that produces a QuotedString or CString has to escape spaces as `\ ` or `\040`.
The reason why RFC 2822 does not require space to be escaped is that the definition of `quoted-string` is not `DQUOTE *qcontent DQUOTE` as in control-spec.txt, but also allows whitespace as part of the `[FWS]` production:
```
quoted-string = [CFWS]
DQUOTE *([FWS] qcontent) [FWS] DQUOTE
[CFWS]
```
I notice that tor doesn't escape the space character (in `esc_for_log` and `unescape_string` for example). IMO tor is doing the right, expected thing and the spec should be clarified.https://gitlab.torproject.org/tpo/core/torspec/-/issues/31Audit list of network parameters for completeness2021-09-16T14:40:39ZNick MathewsonAudit list of network parameters for completeness`dir-spec` has a big list of network parameters, but is it complete? No, it's missing at least ExtendByEd25519. What else might it be missing? We should find out.`dir-spec` has a big list of network parameters, but is it complete? No, it's missing at least ExtendByEd25519. What else might it be missing? We should find out.Nick MathewsonNick Mathewson