Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T16:05:41Zhttps://gitlab.torproject.org/legacy/trac/-/issues/7798Use directory guards even when consensus isn't live2020-06-13T16:05:41ZNick MathewsonUse directory guards even when consensus isn't liveOur initial directory guard implementation only works when we have a live consensus to find out the directory guards' IP addresses. So it's an improvement, but it's hardly a perfect solution.
We should instead remember our guards' IP a...Our initial directory guard implementation only works when we have a live consensus to find out the directory guards' IP addresses. So it's an improvement, but it's hardly a perfect solution.
We should instead remember our guards' IP addresses in our state file, and try to use our directory guards even when we don't have a consensus. This will require consideration about handling our guards being down and handling the network being down.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/18084Use the same fallback directory mirror to bootstrap2020-06-13T14:53:28ZteorUse the same fallback directory mirror to bootstrapIf Tor always tries to use the same fallback directory mirror to bootstrap (when the consensus expires because Tor isn't used), fewer servers learn about Tor clients, and a Tor client is less likely to download from a bad fallback direct...If Tor always tries to use the same fallback directory mirror to bootstrap (when the consensus expires because Tor isn't used), fewer servers learn about Tor clients, and a Tor client is less likely to download from a bad fallback directory mirror.
This is low priority, as most Tor clients only bootstrap once [citation needed].Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/18812[warn] Tried connecting to router at 81.7.17.171:443, but identity key was no...2020-06-13T14:56:22ZRoger Dingledine[warn] Tried connecting to router at 81.7.17.171:443, but identity key was not as expected: wanted 00C4B4731658D3B4987132A3F77100CFCB190D97 but got CFECDDCA990E3EF7B7EC958B22441386B6B8D820.I'm running git master, but I think this applies to Tor 0.2.8.2-alpha. On bootstrap, I got
```
[...]
Apr 13 03:17:50.542 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
...I'm running git master, but I think this applies to Tor 0.2.8.2-alpha. On bootstrap, I got
```
[...]
Apr 13 03:17:50.542 [notice] I learned some more directory information, but not enough to build a circuit: We have no usable consensus.
Apr 13 03:17:50.620 [warn] Tried connecting to router at 81.7.17.171:443, but identity key was not as expected: wanted 00C4B4731658D3B4987132A3F77100CFCB190D97 but got CFECDDCA990E3EF7B7EC958B22441386B6B8D820.
Apr 13 03:17:52.147 [notice] Bootstrapped 40%: Loading authority key certs
[...]
```
So I think everything is going according to plan (cool, I even used a fallback dir!), except my fallback dir seems to have changed its key since it went into fallback_dirs.inc.
Maybe it isn't so reliably stable after all? :) :/
I wonder if we should consider a) doing periodic full checks of the fallback relays, to catch issues like this one preemptively, and then b) making the warning less scary for normal users.Tor: 0.2.8.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/19480Avoid errors during fallback selection when there are no fallbacks2020-06-13T16:05:41ZteorAvoid errors during fallback selection when there are no fallbacksThis issue is fixed as part of #19071, I just needed a bug number.This issue is fixed as part of #19071, I just needed a bug number.Tor: 0.2.8.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/19810Could high-bandwidth ORPort-only relays be fallbacks?2020-06-13T14:59:52ZteorCould high-bandwidth ORPort-only relays be fallbacks?In 0.2.8, relays automatically support begindir over ORPort, even if they don't have a DirPort.
I wonder if we're missing out on (m)any high-bandwidth relays by excluding relays without a DirPort. That said, we wouldn't want too many, b...In 0.2.8, relays automatically support begindir over ORPort, even if they don't have a DirPort.
I wonder if we're missing out on (m)any high-bandwidth relays by excluding relays without a DirPort. That said, we wouldn't want too many, because relays still use the DirPort to bootstrap off fallbacks. (But if the fallback only has an ORPort, the relay will use that.)
We'd need to make the following changes for this to happen:
* modify the DirPort requirement during fallback selection to check for either a DirPort, or declared begindir support in the descriptor
* make a DirPort optional for configured FallbackDirs in Tor
* this may be as simple as setting the DirPort to 0, and disabling the validation. The rest of the code might just work, because it ignores 0 DirPorts
* test, test, testTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/19989Tor fails to bootstrap with an Exit as EntryNode2020-06-13T15:00:41ZteorTor fails to bootstrap with an Exit as EntryNodeWhen I try to run tor with EntryNodes x (where x is a single Exit relay), it hangs at:
```
Aug 26 00:51:57.000 [notice] Bootstrapped 45%: Asking for relay descriptors
Aug 26 00:51:57.000 [notice] I learned some more directory informatio...When I try to run tor with EntryNodes x (where x is a single Exit relay), it hangs at:
```
Aug 26 00:51:57.000 [notice] Bootstrapped 45%: Asking for relay descriptors
Aug 26 00:51:57.000 [notice] I learned some more directory information, but not enough to build a circuit: We need more microdescriptors: we have 0/6973, and can only build 0% of likely paths. (We have 0% of guards bw, 0% of midpoint bw, and 0% of exit bw = 0% of path bw.)
Aug 26 00:51:58.000 [notice] Bootstrapped 50%: Loading relay descriptors
```
~~This issue might prevent us from switching to one guard in future releases.~~
This issue likely makes Exits useless as fallback directories for IPv6-only microdescriptor clients (#19608).
I can reproduce this issue on 0.2.9.2-alpha-dev, 0.2.8.{6,7}, maint-0.2.7, maint-0.2.6 using the following command:
```
src/or/tor DataDirectory /tmp/tor.$$ SOCKSPort 0 EntryNodes x
```Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/20010modifications of relay(s) on fallback whitelist2020-06-13T16:05:42ZTracmodifications of relay(s) on fallback whitelist```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hello teor,
I currently have two relays in the fallback whitelist (niij01 230A8B2A8BA861210D9B4BA97745AEC217A94207 and niij02 0B85617241252517E8ECF2CFC7F4C1A32DCD153F). I have some u...```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hello teor,
I currently have two relays in the fallback whitelist (niij01 230A8B2A8BA861210D9B4BA97745AEC217A94207 and niij02 0B85617241252517E8ECF2CFC7F4C1A32DCD153F). I have some updates/questions:
1. I have added IPv6 addresses for these two relays, is there anything I need to do to update them on the fallback list?
2. I have another fast, stable relay in the family of these 2 relays, that can be added (niij03 A9406A006D6E7B5DA30F2C6D4E42A338B5E340B2) to the fallback.
3. Do the IP addresses for these relays need to remain the same indefinitely? I was thinking of changing hosts (fingerprint would remain the same), but if changing the IP addresses causes issues, I will keep them on their current host.
Thanks, niij
-----BEGIN PGP SIGNATURE-----
wsFcBAEBCAAQBQJXwzUpCRAdSJS4jbcqPQAAsTAQAKs7K1exZHkf8Jyj/sLDBjo+
ZuBTulOQi+PxCstUNZgbOE3xN+LyerrBDBqFLy0znrwj1VK5TgKJi6+EawaJQFWh
qS/Mly8VujsighUdx94vrfxU2AKnvBIQ4oU72+tzXsp7Hsdscr3sG5DOMWTdWNKi
DvK/ccaeCsCkuAsU7UAJ55DtOhtHiJ9fHGMtJYipTXKB/gLUeo8rz5BUyJTGOCOJ
fTWqp1rw+Xbgvo+jPLl8YTsgijA+BMxurCgYng+90VH4P6weZGQFWIn7CQ55ANmO
kRfcw/sSRKXJTYAw6jCNe8eUC8eq1EhfGpbSZoa7KaV7l8UtpEsx7/splUnDtWj6
6KQF9tk+k3YR/2D1oeYfDcyDSJAMXIRH/NLRg7H06vuuoZEQm/Q5lSoZ8whGZbAN
HnKxb66ZNc/RMQ0DgLl1Gs42OMQCLcBsP0I6PFx429TgxnGfnceWpJgEqN0Q9kGy
rJ2J4jBy9kW70Sh813focmVlK3TkkejUcLYoWFz57siqipGY3nsBgtLETHpULEtl
SAhQCs6XjJ9LlRLmXplSj8ftmdTiwvyLKOukbxkrqdEiyDAxS0C9zdSCCfujrqR/
WEyEzbc9hom/Xms2FwCcZ5dFCDbf3CiD722bPbavhGH/6TgAyDzAlqOa2PA1heEr
BDFkOQzVrIyIbnzuoL7S
=0wQz
-----END PGP SIGNATURE-----
```
**Trac**:
**Username**: niijTor: 0.2.9.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20170Backport latest fallback list to 0.2.8 and 0.2.92020-06-13T16:05:43ZteorBackport latest fallback list to 0.2.8 and 0.2.9I'm generating a new list of fallback directory mirrors for release 0.2.9 in #18828.
We have a few options for dealing with the 0.2.8 list:
* comment-out any broken fallbacks,
* backport the 0.2.9 list,
* do nothing.
I would prefer to ...I'm generating a new list of fallback directory mirrors for release 0.2.9 in #18828.
We have a few options for dealing with the 0.2.8 list:
* comment-out any broken fallbacks,
* backport the 0.2.9 list,
* do nothing.
I would prefer to have the same src/or/fallback_dirs.inc in every (relevant) Tor release, assuming we do another 0.2.8 series release.
This is consistent with how we handle directory authorities and geoip.
Otherwise it becomes hard to check multiple fallback lists at once.
But I could be convinced to go with either of the other options.Tor: 0.2.8.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20171Tell 0.2.8 fallback directory operators that their relays are on the list2020-06-13T17:37:18ZteorTell 0.2.8 fallback directory operators that their relays are on the listThank them, and remind them to keep their relay details the same.
I am hoping the community team can help me out with this, if I provide a list.Thank them, and remind them to keep their relay details the same.
I am hoping the community team can help me out with this, if I provide a list.https://gitlab.torproject.org/legacy/trac/-/issues/20172Fallback Tasks for 20162020-06-13T15:01:34ZteorFallback Tasks for 2016This is the top-level task for fallback directory mirror tickets in 2016, typically in 0.2.8 or 0.2.9.This is the top-level task for fallback directory mirror tickets in 2016, typically in 0.2.8 or 0.2.9.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/20173Tell 0.2.9 fallback directory operators that their relays are on the list2020-06-13T16:05:44ZteorTell 0.2.9 fallback directory operators that their relays are on the listThank them, and remind them to keep their relay details the same.
I am hoping the community team can help me out with this, if I provide a list.Thank them, and remind them to keep their relay details the same.
I am hoping the community team can help me out with this, if I provide a list.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20174Automate checking existing fallbacks2020-06-13T16:05:45ZteorAutomate checking existing fallbacksI use a manual process to check existing fallbacks. It would be great if the updateFallbackDirs.py script would automatically read src/or/fallback_dirs.inc, and check each fallback for errors.
For details, see:
https://trac.torproject.o...I use a manual process to check existing fallbacks. It would be great if the updateFallbackDirs.py script would automatically read src/or/fallback_dirs.inc, and check each fallback for errors.
For details, see:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors
I think it can go in 0.3.0Tor: 0.3.0.x-finalhaxxpophaxxpophttps://gitlab.torproject.org/legacy/trac/-/issues/20175Allow the fallback script to ignore temporary IPv6 addresses2020-06-13T16:05:45ZteorAllow the fallback script to ignore temporary IPv6 addressesWhen updateFallbackDirs.py checks relay addresses, it makes sure that the IPv4 and IPv6 addresses and ports match the relay's whitelist entry.
If a relay's IPv6 address is temporary, it should not be included in the whitelist.
But this...When updateFallbackDirs.py checks relay addresses, it makes sure that the IPv4 and IPv6 addresses and ports match the relay's whitelist entry.
If a relay's IPv6 address is temporary, it should not be included in the whitelist.
But this means the relay will never be selected, because its descriptor has an IPv6 address, and that address doesn't match the (missing) address in the whitelist.
We should add a way to say ipv6=ignored or something.Tor: unspecifiedhaxxpophaxxpophttps://gitlab.torproject.org/legacy/trac/-/issues/20177When checking existing fallbacks, report those fallbacks at warning log level2020-06-13T16:05:46ZteorWhen checking existing fallbacks, report those fallbacks at warning log levelWhen the fallback script excludes some relays, it only logs at info level. This is usually what we want, but when checking existing fallbacks, it would be great to see any message about those fallbacks at WARNING log level.
This would m...When the fallback script excludes some relays, it only logs at info level. This is usually what we want, but when checking existing fallbacks, it would be great to see any message about those fallbacks at WARNING log level.
This would make it easier to work out why fallbacks are broken.Tor: unspecifiedhaxxpophaxxpophttps://gitlab.torproject.org/legacy/trac/-/issues/20178The fallback update script should log stem connection errors at warning level2020-06-13T16:05:47ZteorThe fallback update script should log stem connection errors at warning levelCurrently, they're logged at info-level, in order to filter out the "connecting to" messages. It would be great to keep those at info level, and log the "error connecting" messages at WARNING level.Currently, they're logged at info-level, in order to filter out the "connecting to" messages. It would be great to keep those at info level, and log the "error connecting" messages at WARNING level.Tor: unspecifiedhaxxpophaxxpophttps://gitlab.torproject.org/legacy/trac/-/issues/20190Remove broken fallbacks from the 0.2.8 list2020-06-13T16:05:47ZteorRemove broken fallbacks from the 0.2.8 listPlease merge my branch broken-028-fallbacks to both 0.2.8 and master.
It comments-out fallbacks that have broken since I last checked in just before 0.2.8.6.Please merge my branch broken-028-fallbacks to both 0.2.8 and master.
It comments-out fallbacks that have broken since I last checked in just before 0.2.8.6.Tor: 0.2.8.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20192When outputting potential new fallbacks, blacklist the whitelist2020-06-13T16:05:48ZteorWhen outputting potential new fallbacks, blacklist the whitelistWhen we look for potential new fallback directory mirrors, we want to ignore existing whitelisted fallbacks, as well as the blacklist.When we look for potential new fallback directory mirrors, we want to ignore existing whitelisted fallbacks, as well as the blacklist.Tor: unspecifiedhaxxpophaxxpophttps://gitlab.torproject.org/legacy/trac/-/issues/20194Add "update fallbacks" to "Preliminaries" in Tor's doc/HACKING/ReleasingTor.md2020-06-13T15:01:37ZteorAdd "update fallbacks" to "Preliminaries" in Tor's doc/HACKING/ReleasingTor.mdWhen we added fallback directory mirrors, we should have also listed them as something we need to update every major release:
2. Early in the alpha series for each new major release, at least a month before the code freeze, update th...When we added fallback directory mirrors, we should have also listed them as something we need to update every major release:
2. Early in the alpha series for each new major release, at least a month before the code freeze, update the list of fallback directory mirrors using the instructions in: https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrorsTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/20539Make sure fallback directories aren't running buggy versions / can deliver a ...2020-06-13T16:05:49ZteorMake sure fallback directories aren't running buggy versions / can deliver a recent consensusAfter #20499, we should reject fallback directories that deliver a consensus outdated by more than N hours, where N is one of [1, 2, 3].After #20499, we should reject fallback directories that deliver a consensus outdated by more than N hours, where N is one of [1, 2, 3].https://gitlab.torproject.org/legacy/trac/-/issues/20876Avoid contacting fallback operators who are unlikely to be accepted2020-06-13T16:05:49ZteorAvoid contacting fallback operators who are unlikely to be acceptedAfter we automatically calculate the fallback threshold in #20192, it would be great to update that threshold based on whether the operator would be selected if they opted-in.After we automatically calculate the fallback threshold in #20192, it would be great to update that threshold based on whether the operator would be selected if they opted-in.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/20877Fix a bug in updateFallbackDirs.py's comment handling2020-06-13T16:05:50ZteorFix a bug in updateFallbackDirs.py's comment handlingTurns out we weren't returning the comment string. Oops!
Bugfix on 99983432 in tor-0.2.8.3-alpha.Turns out we weren't returning the comment string. Oops!
Bugfix on 99983432 in tor-0.2.8.3-alpha.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20878Add bandwidth to fallback comments2020-06-13T16:05:50ZteorAdd bandwidth to fallback commentsImplemented as part of #18828Implemented as part of #18828teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20880Make minimum fallback stability 6 months2020-06-13T16:05:51ZteorMake minimum fallback stability 6 monthsNow that we've fixed #18050, we can revert the fallback minimum address stability to a longer period. I suggest 6 months - the period between releases.
This will be implemented as part of #18828.Now that we've fixed #18050, we can revert the fallback minimum address stability to a longer period. I suggest 6 months - the period between releases.
This will be implemented as part of #18828.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20881Select 200 fallbacks for each release2020-06-13T16:05:51ZteorSelect 200 fallbacks for each releaseThis allows us to remove fallbacks as needed for 6-12 months, without the list getting too small.
Implemented in #18828.This allows us to remove fallbacks as needed for 6-12 months, without the list getting too small.
Implemented in #18828.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20882Make output sort order of fallbacks configurable2020-06-13T16:05:51ZteorMake output sort order of fallbacks configurableImplemented in #18828.Implemented in #18828.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20883Ignore relays without contact info when emailing potential fallback operators2020-06-13T16:05:52ZteorIgnore relays without contact info when emailing potential fallback operatorsSometimes, we want to do a mail-out to (potential) fallback operators.
We might want to skip operators without contact details.
Or we might want to leave them in there, so we know the how many operators we can't contact.Sometimes, we want to do a mail-out to (potential) fallback operators.
We might want to skip operators without contact details.
Or we might want to leave them in there, so we know the how many operators we can't contact.https://gitlab.torproject.org/legacy/trac/-/issues/20897Make it easier to check the entire fallback whitelist for errors2020-06-13T16:05:52ZteorMake it easier to check the entire fallback whitelist for errorsThere are instructions for modifying the fallback script to scan the entire whitelist in:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors
It would be great if we automated these.There are instructions for modifying the fallback script to scan the entire whitelist in:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors
It would be great if we automated these.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/20908Display the fingerprint when downloading consensuses from fallbacks2020-06-13T15:04:06ZteorDisplay the fingerprint when downloading consensuses from fallbacksJust using this for the bug number, will be fixed in #18828.Just using this for the bug number, will be fixed in #18828.Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/20910Make the fallback list automatically adapt to minor changes2020-06-13T16:05:53ZteorMake the fallback list automatically adapt to minor changesI spend a lot of time contacting relay operators about new and changed IP addresses, fingerprints, etc.
It would be great to identify minor changes (such as adding an IPv6 address) and automatically adapt to them, rather than eliminatin...I spend a lot of time contacting relay operators about new and changed IP addresses, fingerprints, etc.
It would be great to identify minor changes (such as adding an IPv6 address) and automatically adapt to them, rather than eliminating the relay from the list. In the case of an IPv6 address, automatically adding it should be OK.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/20912Allow 2 fallbacks per operator2020-06-13T16:05:54ZteorAllow 2 fallbacks per operatorNow that we are including 200 fallbacks, it seems safe to allow more than one fallback per operator.
Not much point allowing more than one per IP or machine, though.Now that we are including 200 fallbacks, it seems safe to allow more than one fallback per operator.
Not much point allowing more than one per IP or machine, though.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20913Increase fallback stability, flag percentage, and bandwidth2020-06-13T16:05:54ZteorIncrease fallback stability, flag percentage, and bandwidthIn #18828, I had to decrease the fallback criteria to get ~160 fallbacks.
It would be great to increase these to:
* Stability: 3 months+ (from 7 days)
* Flags: 95%+ (from 90%)
* Bandwidth: 2MBytes/s+ (from 1MByte/s)In #18828, I had to decrease the fallback criteria to get ~160 fallbacks.
It would be great to increase these to:
* Stability: 3 months+ (from 7 days)
* Flags: 95%+ (from 90%)
* Bandwidth: 2MBytes/s+ (from 1MByte/s)Tor: 0.3.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20914Consider switching to 3 fallbacks per operator2020-06-13T16:05:55ZteorConsider switching to 3 fallbacks per operatorIt gets us 173 fallbacks, rather than 155.It gets us 173 fallbacks, rather than 155.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20926Avoid checking fallback candidates' DirPorts if they are down in OnionOO2020-06-13T16:05:56ZteorAvoid checking fallback candidates' DirPorts if they are down in OnionOOExclude relays that have been down for 1 or more days from the fallback
candidate list.
When a relay operator has multiple relays, this prioritises relays that are up over relays that are down.
I'll roll this into the #18828 branch.Exclude relays that have been down for 1 or more days from the fallback
candidate list.
When a relay operator has multiple relays, this prioritises relays that are up over relays that are down.
I'll roll this into the #18828 branch.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20934Use standard python multiset implementation in fallback script2020-06-13T16:05:56ZteorUse standard python multiset implementation in fallback scriptI wrote a multiset (bag) implementaion for updateFallbackDirs.py, but python has Counter. I'd rather use someone else's tested code.
https://docs.python.org/2/library/collections.htmlI wrote a multiset (bag) implementaion for updateFallbackDirs.py, but python has Counter. I'd rather use someone else's tested code.
https://docs.python.org/2/library/collections.htmlhttps://gitlab.torproject.org/legacy/trac/-/issues/20939Add average for each flag and uptime to the fallback summary2020-06-13T16:05:57ZteorAdd average for each flag and uptime to the fallback summaryWhen we generate a list of fallbacks, the script outputs a summary of the spread of IP blocks, ports, exit/non-exit, etc.
It would be useful to have the average flags and uptime in this list as well.
We could then do `success_without_a...When we generate a list of fallbacks, the script outputs a summary of the spread of IP blocks, ports, exit/non-exit, etc.
It would be useful to have the average flags and uptime in this list as well.
We could then do `success_without_auth = min(avg_uptime, avg_running, avg_v2dir)^3` and print it too.https://gitlab.torproject.org/legacy/trac/-/issues/20942Make consensus expiry tolerance for fallbacks lower when the stale consensus ...2020-06-13T16:05:57ZteorMake consensus expiry tolerance for fallbacks lower when the stale consensus bug is fixedWhen #20909 is fixed, make CONSENSUS_EXPIRY_TOLERANCE in scripts/maint/updateFallbackDirs.py lower than 24 hours:
* In update_consensus_networkstatus_fetch_time_impl, directory mirrors download consensuses between 2 and 90 minutes after...When #20909 is fixed, make CONSENSUS_EXPIRY_TOLERANCE in scripts/maint/updateFallbackDirs.py lower than 24 hours:
* In update_consensus_networkstatus_fetch_time_impl, directory mirrors download consensuses between 2 and 90 minutes after they are created. So the tolerance can be as low as -90.
* In networkstatus_get_reasonably_live_consensus, REASONABLY_LIVE_TIME is 24 hours after consensus expiry, so the tolerance must not be higher than 24 hours.
* Relay clocks are allowed to drift 12 hours ahead, and 24 hours behind. So it might be reasonable to have the tolerance be 0, or a slightly positive value.
If 0.2.9 is released without #20909 being fixed, wait until most clients are on 0.3.0 before making this change.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20945Avoid an error in the fallback script when a fallback doesn't have any uptime2020-06-13T16:05:58ZteorAvoid an error in the fallback script when a fallback doesn't have any uptimeSometimes, the fallback generation script doesn't add attributes to the
fallbacks in the list. If this happens, log an error, and avoid selecting
that fallback.
Solution in #18828.Sometimes, the fallback generation script doesn't add attributes to the
fallbacks in the list. If this happens, log an error, and avoid selecting
that fallback.
Solution in #18828.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/20946Make it easier to find contacts for a fallback list2020-06-13T16:05:59ZteorMake it easier to find contacts for a fallback listIt would be great to be able to take a list of relays, and find their contact details.
But this might be better implemented using stem, rather than the rather heavyweight updateFallbackDirs.py.
See https://trac.torproject.org/projects/...It would be great to be able to take a list of relays, and find their contact details.
But this might be better implemented using stem, rather than the rather heavyweight updateFallbackDirs.py.
See https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors for detailsTor: 0.3.3.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/20996IPv6-only client fails to bootstrap on microdescriptor fetch/fallback2020-06-13T15:06:30ZmtigasIPv6-only client fails to bootstrap on microdescriptor fetch/fallbackFor a fresh client (nothing cached) when configured with "ClientUseIPv4 0" + "ClientUseIPv6 1" (and client is also on a network with no external IPv4 address).
Hangs at 45%, Asking for relay descriptors. Running on 0.2.9.7-rc.
Works fi...For a fresh client (nothing cached) when configured with "ClientUseIPv4 0" + "ClientUseIPv6 1" (and client is also on a network with no external IPv4 address).
Hangs at 45%, Asking for relay descriptors. Running on 0.2.9.7-rc.
Works fine with UseMicrodescriptors 0. ("ClientPreferIPv6DirPort 1" + "ClientPreferIPv6ORPort 1" have no affect.)
Attaching torrc and a log @ loglevel info.
CCing teor here since I vaguely remember having a conversation with him at last tor meeting where I believe he mentioned to try using "UseMicrodescriptors 0"; which (now that I've had a chance to test) seems like a good workaround at the moment.
Can't seem to find another ticket for this exactly, and just wanted something to refer to (and post some logs and info on), so please excuse if it’s actually a dupe. Possibly related: #20916.
On the mobile side, Apple’s starting to require that apps work in IPv6-only networks. And I’ve noticed that I’ve recently been assigned IPv6/NAT64 (with no external IPv4 address) on T-Mobile lately. But of course this also affects non-mobile IPv6-only networks.
(On the other hand, I'm quite happy that this all _does_ work under IPv6 with the "UseMicrodescriptors 0" workaround.)Tor: 0.3.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21282Adjust fallback log levels2020-06-13T16:06:00ZteorAdjust fallback log levelsThe default mode would be more useful if it issued warnings for changes to fallbacks in the whitelist, and logged (almost) everything else at info level. Then people can make the default log level warning to focus on the most important m...The default mode would be more useful if it issued warnings for changes to fallbacks in the whitelist, and logged (almost) everything else at info level. Then people can make the default log level warning to focus on the most important messages.
The check_existing mode would be more useful if it only logged warning and info messages for relays ~~fingerprints~~ in the existing list. It would also be more useful if it logged a warning message when a fallback in the existing list is missing from OnionOO.https://gitlab.torproject.org/legacy/trac/-/issues/21283Remove broken fallback directory mirrors2020-06-13T15:05:36ZteorRemove broken fallback directory mirrorsSome time around 0.3.0.?-rc, we should remove all non-functional fallback directories from the December 2016 list (or rebuild the entire list).
I'll create a draft branch here, but the script should be re-run just before the release (an...Some time around 0.3.0.?-rc, we should remove all non-functional fallback directories from the December 2016 list (or rebuild the entire list).
I'll create a draft branch here, but the script should be re-run just before the release (and ideally, operators should be contacted and given an opportunity to fix any errors).Tor: 0.3.2.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21564Regenerate fallback list for 0.3.1 or 0.3.22020-06-13T16:06:01ZteorRegenerate fallback list for 0.3.1 or 0.3.2This involves:
* checking for new relays
* contacting potential fallback operators
* rebuilding the list
* contacting all fallback operators
See detailed instructions here:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallb...This involves:
* checking for new relays
* contacting potential fallback operators
* rebuilding the list
* contacting all fallback operators
See detailed instructions here:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrorsTor: 0.3.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/22268TROVE-2017-003: Impersonation of a single fallback directory mirror2020-06-13T15:09:04ZteorTROVE-2017-003: Impersonation of a single fallback directory mirrorSee:
https://lists.torproject.org/pipermail/tor-relays/2017-May/012281.htmlSee:
https://lists.torproject.org/pipermail/tor-relays/2017-May/012281.htmlTor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/22270Fix the usage message in the fallback directory script2020-06-13T15:09:05ZteorFix the usage message in the fallback directory scriptTor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/22271Regenerate fallback list for 0.3.2 or 0.3.32020-06-13T16:06:03ZteorRegenerate fallback list for 0.3.2 or 0.3.3This involves:
* checking for new relays
* contacting potential fallback operators
* rebuilding the list
* contacting all fallback operators
See detailed instructions here:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFal...This involves:
* checking for new relays
* contacting potential fallback operators
* rebuilding the list
* contacting all fallback operators
See detailed instructions here:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrorsTor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/22421Update fallback config comment for exponential backoff2020-06-13T15:09:39ZteorUpdate fallback config comment for exponential backoffWe modified the fallback behaviour when we merged the exponential backoff code in 0.2.9.1-alpha, and again in #17750:
```
/* With the ClientBootstrapConsensus*Download* below:
* Clients with only authorities will try:
* - 3 author...We modified the fallback behaviour when we merged the exponential backoff code in 0.2.9.1-alpha, and again in #17750:
```
/* With the ClientBootstrapConsensus*Download* below:
* Clients with only authorities will try:
* - 3 authorities over 10 seconds, then wait 60 minutes.
* Clients with authorities and fallbacks will try:
* - 2 authorities and 4 fallbacks over 21 seconds, then wait 60 minutes.
* Clients will also retry when an application request arrives.
* After a number of failed reqests, clients retry every 3 days + 1 hour.
*
* Clients used to try 2 authorities over 10 seconds, then wait for
* 60 minutes or an application request.
*
* When clients have authorities and fallbacks available, they use these
* schedules: (we stagger the times to avoid thundering herds) */
V(ClientBootstrapConsensusAuthorityDownloadSchedule, CSV_INTERVAL,
"6, 11, 3600, 10800, 25200, 54000, 111600, 262800" /* 3 days + 1 hour */),
V(ClientBootstrapConsensusFallbackDownloadSchedule, CSV_INTERVAL,
"0, 1, 4, 11, 3600, 10800, 25200, 54000, 111600, 262800"),
/* When clients only have authorities available, they use this schedule: */
V(ClientBootstrapConsensusAuthorityOnlyDownloadSchedule, CSV_INTERVAL,
"0, 3, 7, 3600, 10800, 25200, 54000, 111600, 262800"),
```
The behaviour is now:
```
/* With the ClientBootstrapConsensus*Download* below:
* Clients with only authorities will try:
* - at least 3 authorities over 10 seconds, then exponentially backoff,
* with the next attempt 3-21 seconds later,
* Clients with authorities and fallbacks will try:
* - at least 2 authorities and 4 fallbacks over 21 seconds, then
* exponentially backoff, with the next attempts 4-33 seconds later,
* Clients will also retry when an application request arrives.
* After a number of failed requests, clients retry every 3 days + 1 hour.
*
* Clients used to try 2 authorities over 10 seconds, then wait for
* 60 minutes or an application request.
```Tor: 0.3.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/22527Add new operators to fallback directory whitelist2020-06-13T16:06:07ZteorAdd new operators to fallback directory whitelistI will keep a list here, and do them all at once.I will keep a list here, and do them all at once.Tor: 0.3.3.x-finalpastlypastlyhttps://gitlab.torproject.org/legacy/trac/-/issues/22760Fix extra-info flags on fallbacks2020-06-13T15:10:54ZteorFix extra-info flags on fallbacksSee the first comment for details.See the first comment for details.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/23471Add ed25519 ids to torrc relay configs2020-06-13T15:13:31ZteorAdd ed25519 ids to torrc relay configsTor relays have ed25519 keys, but there is no way to add them to the torrc (or the hard-coded configs compiled into tor).
Tor accepts at least 3 types of hard-coded relay lists in its configs:
* Directory Authorities
* Fallback Director...Tor relays have ed25519 keys, but there is no way to add them to the torrc (or the hard-coded configs compiled into tor).
Tor accepts at least 3 types of hard-coded relay lists in its configs:
* Directory Authorities
* Fallback Directory Mirrors
* BridgesTor: 0.3.3.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/23473Add support for ed25519 fallbacks ids to the fallback script2020-06-13T16:06:09ZteorAdd support for ed25519 fallbacks ids to the fallback scripthttps://gitlab.torproject.org/legacy/trac/-/issues/24600Add fallback nicknames to the file, so stem can use them2020-06-13T16:06:10ZteorAdd fallback nicknames to the file, so stem can use themWe should do this in a machine-readable comment, to avoid inflating the tor binary with useless data.We should do this in a machine-readable comment, to avoid inflating the tor binary with useless data.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24678Update fallback directory whitelist based on latest relay changes2020-06-13T16:06:11ZteorUpdate fallback directory whitelist based on latest relay changesWe need to run the script, and get the list of warnings about changed relay details.
Then we can contact each operator to see if the change is permanent.We need to run the script, and get the list of warnings about changed relay details.
Then we can contact each operator to see if the change is permanent.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24679Remove custom weights from each fallback in the fallback list2020-06-13T16:06:12ZteorRemove custom weights from each fallback in the fallback listThis makes the fallback list 1 byte smaller per fallback, or ~150 bytes.This makes the fallback list 1 byte smaller per fallback, or ~150 bytes.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24681Make the default fallback weight in Tor 10.02020-06-13T15:19:15ZteorMake the default fallback weight in Tor 10.0This is a follow-up to #24679.
* update the default weight in parse_dir_fallback_line() to 10.0
* update the man page to reflect the new default
10.0 gives us:
* 0.5% clients bootstrapping off an authority when all fallbacks are up, * ...This is a follow-up to #24679.
* update the default weight in parse_dir_fallback_line() to 10.0
* update the man page to reflect the new default
10.0 gives us:
* 0.5% clients bootstrapping off an authority when all fallbacks are up, * 2% when 25% are down (our replacement threshold)
* 7% when 40% are down (our worst case scenario)Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24682Make fallback script usage instructions create a log file2020-06-13T16:06:13ZteorMake fallback script usage instructions create a log fileThis is a comment-only change.This is a comment-only change.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24708Make the fallback script search harder for python2020-06-13T16:06:15ZteorMake the fallback script search harder for pythonSome OSs don't put python in /usr/bin. pastly will tell you this.Some OSs don't put python in /usr/bin. pastly will tell you this.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24711Update dependency list for the fallback script2020-06-13T16:06:16ZteorUpdate dependency list for the fallback scriptThis is a comment-only changeThis is a comment-only changeTor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24719Make sure fallback whitelist entries are unique2020-06-13T16:06:16ZteorMake sure fallback whitelist entries are uniqueWe shouldn't be duplicating fingerprint, or any address/port combination.
But there's no urgency here, it's mostly harmless, it just means we won't catch relays that switch details.We shouldn't be duplicating fingerprint, or any address/port combination.
But there's no urgency here, it's mostly harmless, it just means we won't catch relays that switch details.https://gitlab.torproject.org/legacy/trac/-/issues/24725Add a format version number to the fallback file2020-06-13T16:06:17ZteorAdd a format version number to the fallback fileatagar asked for this, and it's a good idea.
Let's use semantic versioning:
* major versions are for incompatible changes, like removing non-optional fields
* minor versions are for compatible changes, like adding fields
* patch version...atagar asked for this, and it's a good idea.
Let's use semantic versioning:
* major versions are for incompatible changes, like removing non-optional fields
* minor versions are for compatible changes, like adding fields
* patch versions are for bug fixes, like changing an incorrectly-formatted header commentTor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24726Make sure a comma never appears anywhere in a fallback entry2020-06-13T16:06:17ZteorMake sure a comma never appears anywhere in a fallback entryThis possibly requires an update to our strong cleansing routines, or just adding a comment to them.This possibly requires an update to our strong cleansing routines, or just adding a comment to them.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24742Add fallback list spec to torspec2020-06-13T16:06:18ZteorAdd fallback list spec to torspecPlease see my torspec branch fallback-format-2 on github, which adds the fallback-spec.txt specification.
This was requested by irl on tor-dev@, so that metrics-lib could have some assurance of the file format. I'm sure it will help ste...Please see my torspec branch fallback-format-2 on github, which adds the fallback-spec.txt specification.
This was requested by irl on tor-dev@, so that metrics-lib could have some assurance of the file format. I'm sure it will help stem, too.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24785Reduce the fallback stability and flag requirements due to extra network load2020-06-13T16:06:19ZteorReduce the fallback stability and flag requirements due to extra network loadWe're only getting 700 candidates, we should adjust the settings so we get about 1500, like last time.We're only getting 700 candidates, we should adjust the settings so we get about 1500, like last time.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24786Rebuild the fallback list in 20182020-06-13T16:21:09ZteorRebuild the fallback list in 2018We need to rebuild the list of fallbacks in late 2018 or early 2019.
We usually do this when 25% or more go down.
(This is tracked in #tor-bots on IRC.)
Here are the instructions for running a rebuild:
https://trac.torproject.org/projec...We need to rebuild the list of fallbacks in late 2018 or early 2019.
We usually do this when 25% or more go down.
(This is tracked in #tor-bots on IRC.)
Here are the instructions for running a rebuild:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrorsTor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/24787Revise the fallback stability requirements based on current network load2020-06-13T16:06:21ZteorRevise the fallback stability requirements based on current network loadIn #24785, we decreased the fallback stability requirements due to the increased network load from December 2017.
We should work out whether we can raise them again.In #24785, we decreased the fallback stability requirements due to the increased network load from December 2017.
We should work out whether we can raise them again.https://gitlab.torproject.org/legacy/trac/-/issues/24789Consider changing the fallback process to opt-out2020-06-13T16:06:22ZteorConsider changing the fallback process to opt-outIf we did this, we could rebuild the list by running a script, and occasionally collect opt-outs and add them to the blacklist.
I don't think this would worry anyone too much.If we did this, we could rebuild the list by running a script, and occasionally collect opt-outs and add them to the blacklist.
I don't think this would worry anyone too much.Tor: 0.3.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/24790When 0.2.5 is EOL, remove the fallback script comment that refers to 0.2.8 an...2020-06-13T16:06:22ZteorWhen 0.2.5 is EOL, remove the fallback script comment that refers to 0.2.8 and earlierWe don't accept unsupported Tor versions as fallbacks, so there's no reason to talk about other bugs in unsupported versions.We don't accept unsupported Tor versions as fallbacks, so there's no reason to talk about other bugs in unsupported versions.Tor: 0.3.4.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/24791Remove excessive address info from the fallback script log2020-06-13T16:06:23ZteorRemove excessive address info from the fallback script logThis makes it easier for people to focus on the info that actually matters. Also, it makes it fit in a single terminal window.This makes it easier for people to focus on the info that actually matters. Also, it makes it fit in a single terminal window.Tor: 0.3.3.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24801Generate a new fallback list and backport it2020-06-13T16:06:24ZteorGenerate a new fallback list and backport itI can generate a new list over the weekend.
This will be easier once all the other children of #22271 merge.I can generate a new list over the weekend.
This will be easier once all the other children of #22271 merge.Tor: 0.3.2.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24803Generate a new fallback list in 2018 and backport it to all supported versions2020-06-13T15:19:50ZteorGenerate a new fallback list in 2018 and backport it to all supported versionsThis is the actual list generation ticket.This is the actual list generation ticket.Tor: 0.2.9.x-finalColin ChildsColin Childshttps://gitlab.torproject.org/legacy/trac/-/issues/24804Run an opt-in process for relay operators to become fallbacks in 20182020-06-13T16:06:25ZteorRun an opt-in process for relay operators to become fallbacks in 2018This involves mailing tor-relays and asking if stable relay operators want to become fallbacks.This involves mailing tor-relays and asking if stable relay operators want to become fallbacks.Tor: 0.4.0.x-finalColin ChildsColin Childshttps://gitlab.torproject.org/legacy/trac/-/issues/24805Update fallback whitelist in late 20182020-06-13T15:19:52ZteorUpdate fallback whitelist in late 2018We need to update the list using:
* operator emails from 2018
* the opt-in process in #24804
* ~~running the script, finding change warnings, contacting operators, and updating their details~~ we'll do #24838 insteadWe need to update the list using:
* operator emails from 2018
* the opt-in process in #24804
* ~~running the script, finding change warnings, contacting operators, and updating their details~~ we'll do #24838 insteadTor: 0.4.0.x-finalColin ChildsColin Childshttps://gitlab.torproject.org/legacy/trac/-/issues/24838Fuzzy match the fallback whitelist2020-06-13T15:20:06ZteorFuzzy match the fallback whitelistWe can rely on Onionoo to tell us when relay fingerprints or addresses change, so we don't need to do an exact match.
But listing fingerprints, addresses and ports in the whitelist is useful so that we know what a relay operator origina...We can rely on Onionoo to tell us when relay fingerprints or addresses change, so we don't need to do an exact match.
But listing fingerprints, addresses and ports in the whitelist is useful so that we know what a relay operator originally asked for.
Fuzzy matching simplifies maintaining the fallback whitelist, because we don't have to ask operators to update their relay details. (Or ask if those details are permanent.)
We deleted the blacklist in #26502.Tor: 0.4.0.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/24839Add a torrc option and descriptor line to opt-in as a FallbackDir2020-06-13T16:06:51ZteorAdd a torrc option and descriptor line to opt-in as a FallbackDirThis needs:
* a proposal and a design
* a patch to dir-spec.txt
* a patch to the tor man page
* a tor code patch
* an updateFallbackDirs.py code patch
* a wiki update to [[doc/UpdatingFallbackDirectoryMirrors]]
Here's a quick sketch of ...This needs:
* a proposal and a design
* a patch to dir-spec.txt
* a patch to the tor man page
* a tor code patch
* an updateFallbackDirs.py code patch
* a wiki update to [[doc/UpdatingFallbackDirectoryMirrors]]
Here's a quick sketch of a design:
1. Relay operators set `OfferFallbackDir 1` to offer their relay as a potential FallbackDir.
2. Relays with this option set put `offer-fallback-dir` in their descriptors
3. updateFallbackDirs.py loads relay fingerprints with `offer-fallback-dir`, and from the legacy offer list
4. updateFallbackDirs.py does stability checks, and generates the fallback listTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/24953In check_existing mode, log "fallback list", not "whitelist"2020-06-13T16:06:26ZteorIn check_existing mode, log "fallback list", not "whitelist"This confused at least one relay operator.This confused at least one relay operator.Tor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/25798Remove from fallback list:2020-06-13T16:06:29ZTracRemove from fallback list:Relay treadstone (185.129.249.124) has been shut down by my provider and unlikely to be back (the website of the provider, including the control panel has been gone for days). Please remove from the fallback list.
**Trac**:
**Username...Relay treadstone (185.129.249.124) has been shut down by my provider and unlikely to be back (the website of the provider, including the control panel has been gone for days). Please remove from the fallback list.
**Trac**:
**Username**: iomegaTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26502Stop using the fallback blacklist, and delete it2020-06-13T16:06:29ZteorStop using the fallback blacklist, and delete itWe require relay operators to opt-in to being fallbacks, and we won't ever switch to opt-out. (See #24789.)
So we don't need the fallback blacklist any more.
We should stop loading the blacklist in the script, then delete it.We require relay operators to opt-in to being fallbacks, and we won't ever switch to opt-out. (See #24789.)
So we don't need the fallback blacklist any more.
We should stop loading the blacklist in the script, then delete it.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/26685Add ed25519 id support for the hard-coded fallback and authority lists2020-06-13T15:27:41ZteorAdd ed25519 id support for the hard-coded fallback and authority listsThis is the parent ticket for getting ed25519 id support in Tor's hard-coded directory lists.This is the parent ticket for getting ed25519 id support in Tor's hard-coded directory lists.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/26686Add ed25519 ids to the fallback whitelist2020-06-13T16:06:30ZteorAdd ed25519 ids to the fallback whitelistThe fallback scripts should parse ed25519 ids from the fallback whitelist, and check both the RSA id and ed25519 id.The fallback scripts should parse ed25519 ids from the fallback whitelist, and check both the RSA id and ed25519 id.https://gitlab.torproject.org/legacy/trac/-/issues/26687Output ed25519 IDs in the authority and fallback lists2020-06-13T16:06:30ZteorOutput ed25519 IDs in the authority and fallback listshttps://gitlab.torproject.org/legacy/trac/-/issues/26688Parse ed25519 IDs in the authority and fallback lists2020-06-13T15:27:41ZteorParse ed25519 IDs in the authority and fallback listsTor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27737Manually remove 64.113.32.29 from the generated fallback list2020-06-13T16:06:31ZteorManually remove 64.113.32.29 from the generated fallback listIf 64.113.32.29 hasn't changed to 198.232.165.2 yet, we should manually remove it from the generated fallback list.If 64.113.32.29 hasn't changed to 198.232.165.2 yet, we should manually remove it from the generated fallback list.Tor: 0.3.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/27914Extract fallback-scripts to its own git repository2020-06-13T16:06:32ZNick MathewsonExtract fallback-scripts to its own git repositoryThis would let us give teor and phoul direct commit permissions here.This would let us give teor and phoul direct commit permissions here.Tor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/28768Update fallback script to match Tor bootstrap changes2020-06-13T16:06:33ZteorUpdate fallback script to match Tor bootstrap changesIn #24661, we make clients bootstrap from reasonably old consensuses (expired in the last 24 hours).
In #28591, we make clients accept future consensuses as long as they're reasonably future consensuses (valid up to 24 hours in the futu...In #24661, we make clients bootstrap from reasonably old consensuses (expired in the last 24 hours).
In #28591, we make clients accept future consensuses as long as they're reasonably future consensuses (valid up to 24 hours in the future), and relays serve those consensuses.
We need to change the fallback checks to match these changes in Tor.Tor: 0.4.0.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28793Rebuild the fallback list in 20192020-06-13T16:06:34ZteorRebuild the fallback list in 2019We need to rebuild the list of fallbacks in mid or late 2019.
We usually do this when 25% or more go down.
(When 25% of fallbacks are down, a warning is logged daily in #tor-bots on IRC.)
Here are the instructions for running a rebuild:...We need to rebuild the list of fallbacks in mid or late 2019.
We usually do this when 25% or more go down.
(When 25% of fallbacks are down, a warning is logged daily in #tor-bots on IRC.)
Here are the instructions for running a rebuild:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrorsTor: unspecifiedteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28794Run an opt-in process for relay operators to become fallbacks in 20192020-06-13T16:06:36ZteorRun an opt-in process for relay operators to become fallbacks in 2019Mail tor-relays and asking if stable relay operators want to become fallbacks. Then update fallback whitelist based on the opt-ins.Mail tor-relays and asking if stable relay operators want to become fallbacks. Then update fallback whitelist based on the opt-ins.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28795Generate a new fallback list in 2019 and backport it to all supported Tor ver...2020-06-13T15:35:26ZteorGenerate a new fallback list in 2019 and backport it to all supported Tor versionsWe usually get two people to run:
```
scripts/maint/updateFallbackDirs.py > src/or/fallback_dirs.inc 2> fallback_dirs.log
```
from two different locations, then merge the lists.
Please attach the logs to this ticket.We usually get two people to run:
```
scripts/maint/updateFallbackDirs.py > src/or/fallback_dirs.inc 2> fallback_dirs.log
```
from two different locations, then merge the lists.
Please attach the logs to this ticket.Tor: 0.2.9.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/28796Move the fallback script and whitelist into its own repository2020-06-13T16:06:36ZteorMove the fallback script and whitelist into its own repositoryWe'd like to move the fallback script and whitelist into its own repository. Then we can make changes as needed, rather than queueing changes on large branches.
We can also set up CI to test the script with a small number of relays.We'd like to move the fallback script and whitelist into its own repository. Then we can make changes as needed, rather than queueing changes on large branches.
We can also set up CI to test the script with a small number of relays.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28797Set up CI on the fallback script with a small number of relays2020-06-13T16:06:37ZteorSet up CI on the fallback script with a small number of relaysWe should test the list generation and check_existing modes, with a few relays (10?).We should test the list generation and check_existing modes, with a few relays (10?).https://gitlab.torproject.org/legacy/trac/-/issues/28986Log at info level in list generation mode2020-06-13T16:06:39ZteorLog at info level in list generation modestarlight suggests that we log at info level by default:
https://trac.torproject.org/projects/tor/ticket/28795#comment:4
We can make the same change in check_existing mode if we like.starlight suggests that we log at info level by default:
https://trac.torproject.org/projects/tor/ticket/28795#comment:4
We can make the same change in check_existing mode if we like.https://gitlab.torproject.org/legacy/trac/-/issues/29093Announce the new fallback list, and tell downstream maintainers that it has c...2020-06-13T16:06:40ZteorAnnounce the new fallback list, and tell downstream maintainers that it has changedSteps 4 & 5 of:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors#ATypicalReleaseSteps 4 & 5 of:
https://trac.torproject.org/projects/tor/wiki/doc/UpdatingFallbackDirectoryMirrors#ATypicalReleaseTor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/29100Update src/app/config/fallback_dirs.inc to ../tor/src/app/config/fallback_dir...2020-06-13T16:06:41ZteorUpdate src/app/config/fallback_dirs.inc to ../tor/src/app/config/fallback_dirs.inc post-splitThere are a bunch of references to `src/app/config/fallback_dirs.inc` for check_existing mode, and in the comments.
We should update them to `../tor/src/app/config/fallback_dirs.inc` post-split.There are a bunch of references to `src/app/config/fallback_dirs.inc` for check_existing mode, and in the comments.
We should update them to `../tor/src/app/config/fallback_dirs.inc` post-split.https://gitlab.torproject.org/legacy/trac/-/issues/29101Configure the push hook from git.tpo to github for fallback-scripts2020-06-13T16:56:17ZteorConfigure the push hook from git.tpo to github for fallback-scriptsWe need to:
* create a repository for fallback-scripts on https://github.com/torproject
* give phoul and teor and network-team and the pusher permissions
* configure the pusher on the git.torproject.org endWe need to:
* create a repository for fallback-scripts on https://github.com/torproject
* give phoul and teor and network-team and the pusher permissions
* configure the pusher on the git.torproject.org endTor: 0.4.0.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/29103Add a licence, readme, and code of conduct to fallback-scripts2020-06-13T16:06:42ZteorAdd a licence, readme, and code of conduct to fallback-scriptsWe should use the Tor licence, because that's what the original code was committed under. Same with the code of conduct. The README might take more work.We should use the Tor licence, because that's what the original code was committed under. Same with the code of conduct. The README might take more work.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/30785Give Gus access to the fallback-scripts repository2020-06-13T16:57:55ZteorGive Gus access to the fallback-scripts repository```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi,
Please give gus push access to the fallback-scripts repository at:
https://gitweb.torproject.org/fallback-scripts.git
Thanks!
teor, Thu 6 Jun 2019 08:14:29 UTC
-----BEGIN PGP S...```
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA256
Hi,
Please give gus push access to the fallback-scripts repository at:
https://gitweb.torproject.org/fallback-scripts.git
Thanks!
teor, Thu 6 Jun 2019 08:14:29 UTC
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEEo9HIo7IdGQ1wBWG3EP6qDnB1ZyoFAlz4y4cACgkQEP6qDnB1
Zyp3Ag//R5/VzuzKAhOn5zlk9YNfUlVbgcD50NZqRqs73gYXAojAR+GrDMPQe/D+
RvY2qgR/4UQn4kdxt9zZahMn7iePwHADWsDyfdxAkdK9qsdPOL0P9OES5ahJITiA
dFMQiQfweR+b+oyYQTaYWKrDzj7hvWKAk+KxmRTbMyvMxpVfqFMtQZQg+xLsDxl2
xI0PXRs0wAxsNz5+WO0ySPHoc9yZo9Qm1Hw709O6wGgmQOeHh5LmyNjIcOzWsleO
MfCgeVPdE9xGHf5CRzM3Vej2JBeihGr+wDsQMS6myKrlXOjxVd/50vybyZsZAkaC
n43ObaYwvXuvuxARgQQcWc0ICNdJXr5z/0HH1PC9AWqxMNd/9whYKlXWySL1mKfh
aRVF9byoqL3kuAS9PXkpAAN1PrhOtLMHGOrLCeUHdDiHlu5foMq5YkxBOK5RNqHv
n7xl8aWorwwAwyHf7Qw5AytVof5xaDnvthV67W0D4AlgumVZNwRSA3IJWTxKHrGE
U5XZJa//U16yH5y/rBajMAWuVcPzzHm49msQao8vY+ZXWJs5OvccP8NfA4kMZdtB
z4qIb5iwCaZAb3pIIjwx9B7DKaydW/dmLsLpUsykA7H+joUKF+rTnYJiD5JjA48v
NaXBIBBIRPmc9A11OX1WJHK5H1YKtFCk9UYq7xlyMASITIgWNrs=
=RpNx
-----END PGP SIGNATURE-----
```https://gitlab.torproject.org/legacy/trac/-/issues/30947Add a source line to the header, because type must always be fallback2020-06-13T16:06:44ZteorAdd a source line to the header, because type must always be fallbackThe dir list spec says that fallback files start with:
`/* type=fallback */`
https://gitweb.torproject.org/torspec.git/tree/dir-list-spec.txt#n140
But in #24953, we change type to whitelist by default. This might cause some parsers to b...The dir list spec says that fallback files start with:
`/* type=fallback */`
https://gitweb.torproject.org/torspec.git/tree/dir-list-spec.txt#n140
But in #24953, we change type to whitelist by default. This might cause some parsers to break.
Instead, we should add a new `/* source=whitelist|fallback*/` line.Tor: 0.4.1.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/30948Copy the relevant parts of .gitignore to fallback-scripts2020-06-13T16:06:44ZteorCopy the relevant parts of .gitignore to fallback-scriptsteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/30949Add the source= line to the dir list spec2020-06-13T15:42:50ZteorAdd the source= line to the dir list specIn #30947, we added a source line to the fallback file header.
Now we need to update the spec.In #30947, we added a source line to the fallback file header.
Now we need to update the spec.Tor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/30951Follow up broken relays from fallback opt-ins2020-06-13T16:06:45ZteorFollow up broken relays from fallback opt-insThese relays couldn't be added to the fallback whitelist:
```
## Relays that need follow up ##
# https://lists.torproject.org/pipermail/tor-relays/2019-May/017325.html
# 2206C72ECC0D55593BC7B698F982533F1E141DD2 not found in recent descr...These relays couldn't be added to the fallback whitelist:
```
## Relays that need follow up ##
# https://lists.torproject.org/pipermail/tor-relays/2019-May/017325.html
# 2206C72ECC0D55593BC7B698F982533F1E141DD2 not found in recent descriptors
# Email sent directly to gus
# AFD1E28D6BFDFF03E715AF06259167ADA0E0CB1D not found in recent descriptors
# https://lists.torproject.org/pipermail/tor-relays/2019-June/017393.html
# A85FF376759C994A8A1168D8D8219C8C43F6C5E1 not found in recent descriptors
# https://lists.torproject.org/pipermail/tor-relays/2019-June/017394.html
# A850B6A31ED83FB92B34FB3AE0513902D053A1C8 needs a DirPort
# https://lists.torproject.org/pipermail/tor-relays/2019-June/017395.html
# E8D114B3C78D8E6E7FEB1004650DD632C2143C9E needs a DirPort
# https://lists.torproject.org/pipermail/tor-relays/2019-June/017398.html
# C6B656BA6BC16E31115A1B2D56396A53165F3408 needs a DirPort
```teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/30952Work out why the latest fallback list only has 127 entries2020-06-13T16:06:45ZteorWork out why the latest fallback list only has 127 entriesWe seem to have dropped 25 since last time, which isn't great.
I should also look at the long-term number of fallbacks.We seem to have dropped 25 since last time, which isn't great.
I should also look at the long-term number of fallbacks.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/30955Update the fallback entry in the man page2020-06-13T15:42:52ZteorUpdate the fallback entry in the man page"FallbackDir ipv4address:port"
"FallbackDir ipv4address:dirport"
"DirAuthority [nickname] [flags] ipv4address:port"
"DirAuthority [nickname] [flags] ipv4address:dirport"
And similarly for orport.
"The provided port value is a dirport;..."FallbackDir ipv4address:port"
"FallbackDir ipv4address:dirport"
"DirAuthority [nickname] [flags] ipv4address:port"
"DirAuthority [nickname] [flags] ipv4address:dirport"
And similarly for orport.
"The provided port value is a dirport; clients ignore this in favor of the specified "orport=" value."
"Clients always use the orport. Relays prefer the dirport, but will use the orport in some circumstances."
Add something to the FallbackDir entry talking about how the DirPort is used by the checking script?Tor: 0.4.2.x-finalteorteor