Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T17:48:14Zhttps://gitlab.torproject.org/legacy/trac/-/issues/33269Prop 313: 8.1. Check IPv6 Relay Consensus Counts Script2020-06-13T17:48:14ZteorProp 313: 8.1. Check IPv6 Relay Consensus Counts ScriptReview the IPv6 relay count script from proposal 313 (#33262).
There isn't really much to review here. There are only 4 calculations. We've done counts like this before.
See proposal 313, section 8.1, metrics review part:
https://git...Review the IPv6 relay count script from proposal 313 (#33262).
There isn't really much to review here. There are only 4 calculations. We've done counts like this before.
See proposal 313, section 8.1, metrics review part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n337https://gitlab.torproject.org/legacy/trac/-/issues/33270Prop 313: 8.1. Monitor IPv6 Relay Counts in the Consensus2020-06-13T15:51:17ZteorProp 313: 8.1. Monitor IPv6 Relay Counts in the ConsensusMonitor the output of the script in #33262.
Continue monitoring as more volunteer relay operators deploy the relevant tor versions. (And as the number of IPv6 relays in the consensus increases.)
There isn't actually much to check here....Monitor the output of the script in #33262.
Continue monitoring as more volunteer relay operators deploy the relevant tor versions. (And as the number of IPv6 relays in the consensus increases.)
There isn't actually much to check here. If we do find issues, we should open separate bugs for them.
See proposal 313, section 8.1, ongoing monitoring part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n337Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33271Prop 313: 8.2. Test IPv6 Statistics using Chutney2020-06-13T15:50:22ZteorProp 313: 8.2. Test IPv6 Statistics using ChutneyWe want to test IPv6 connection and consumed bandwidth statistics using
chutney networks. However, chutney runs for a short amount of time, and
creates a limited amount of traffic, so we also need to test these changes
on the public tor ...We want to test IPv6 connection and consumed bandwidth statistics using
chutney networks. However, chutney runs for a short amount of time, and
creates a limited amount of traffic, so we also need to test these changes
on the public tor network.
In particular, we have struggled to test statistics using chutney, because
tor's hard-coded statistics period is 24 hours. (And most chutney networks
run for under 1 minute.)
Maybe we can change the statistics interval to 10-30 seconds, and see if that works?
See proposal 313, section 8.2, chutney tests part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n355Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33272Prop 313: 8.2. Test IPv6 Stats on the Tor Network2020-06-13T15:51:18ZteorProp 313: 8.2. Test IPv6 Stats on the Tor NetworkWe want to test the IPv6 connection and bandwidth statistics on the public network, with a small number of relays and bridges.
See proposal 313, section 8.2, public tor network tests part:
https://gitweb.torproject.org/torspec.git/tree/...We want to test the IPv6 connection and bandwidth statistics on the public network, with a small number of relays and bridges.
See proposal 313, section 8.2, public tor network tests part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n355Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33273Prop 313: 8.2. Analyse and Monitor IPv6 Stats2020-06-13T17:48:14ZteorProp 313: 8.2. Analyse and Monitor IPv6 StatsAs part of Sponsor 55, the Tor Metrics team will analyse the new IPv6 connection and bandwidth statistics.
We will use their analysis to provide IPv6 progress reports.
We might also discover some bugs in tor's IPv6 stats code during th...As part of Sponsor 55, the Tor Metrics team will analyse the new IPv6 connection and bandwidth statistics.
We will use their analysis to provide IPv6 progress reports.
We might also discover some bugs in tor's IPv6 stats code during the first analysis.
See proposal 313, section 8.2, metrics analysis part:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n355
The definitions of these statistics are in proposal 313, sections 4 and 5:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n118https://gitlab.torproject.org/legacy/trac/-/issues/33280Add a "make test-network-ipv6" target, which runs IPv6 chutney networks2020-06-13T15:51:19ZteorAdd a "make test-network-ipv6" target, which runs IPv6 chutney networksPart of #33195.Part of #33195.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33333Add a mixed+hs-v23-ipv6 network to chutney2020-06-13T15:51:26ZteorAdd a mixed+hs-v23-ipv6 network to chutneyWe want to test our new IPv6 code in mixed networks, so let's add a `mixed+hs-v23-ipv6` network to chutney.
It should be pretty much a copy-paste from `mixed+hs-v23` and `hs-v23-ipv6-md`.
(There's no need for a non-md variant, all supp...We want to test our new IPv6 code in mixed networks, so let's add a `mixed+hs-v23-ipv6` network to chutney.
It should be pretty much a copy-paste from `mixed+hs-v23` and `hs-v23-ipv6-md`.
(There's no need for a non-md variant, all supported tor versions can use microdescriptors for IPv6 onion services.)
Proposed design:
* 2 dual-stack authorities,
* 2 IPv4-only relays,
* 2 dual-stack relays
* 2 IPv6-only v2 onion services
* 2 IPv6-only v3 onion services
* 2 IPv6-only clients
For each kind of node, one should be a new version, and one should be an old version.
Other networks test IPv4-only authorities and clients, so we don't need them. (But we want to test IPv4 and dual-stack relays in the same network.)Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33334Add a mixed+hs-v23-ipv6 network to tor's test-network2020-06-13T15:51:26ZteorAdd a mixed+hs-v23-ipv6 network to tor's test-networkWe want to add the new mixed+hs-v23-ipv6 chutney network from #33333 to tor's makefile tests:
* test-network-all
* test-network-ipv6
We'll need some minor refactoring, so that we can test for "ipv6" and "mixed", before using this networ...We want to add the new mixed+hs-v23-ipv6 chutney network from #33333 to tor's makefile tests:
* test-network-all
* test-network-ipv6
We'll need some minor refactoring, so that we can test for "ipv6" and "mixed", before using this network.
I think the resulting code will be simpler.
We can pass the following network lists to the test-network-run target:
* unconditional (ipv4 and not mixed)
* ipv6
* mixed
* mixed_ipv6
Then we can:
* test for ipv6 and mixed in separate tests,
* set flags for mixed and ipv6, and
* skip or use the right network lists.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33375Stop advertising an IPv6 exit policy when DNS is broken for IPv62020-06-13T15:51:37ZteorStop advertising an IPv6 exit policy when DNS is broken for IPv6When `dns_seems_to_be_broken_for_ipv6()`, exits should stop advertising an IPv6 exit policy.
Here's a rough design:
* when `dns_seems_to_be_broken_for_ipv6()` is first set to 1, mark the relay descriptor dirty
* when rebuilding the desc...When `dns_seems_to_be_broken_for_ipv6()`, exits should stop advertising an IPv6 exit policy.
Here's a rough design:
* when `dns_seems_to_be_broken_for_ipv6()` is first set to 1, mark the relay descriptor dirty
* when rebuilding the descriptor, check `dns_seems_to_be_broken_for_ipv6()` before including an IPv6 exit policy
* reset `dns_seems_to_be_broken_for_ipv6()` periodically, maybe every 1-3 days?Tor: 0.4.4.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33376Update the networks in Chutney's CI to match Tor's new test-network*2020-06-13T13:31:33ZteorUpdate the networks in Chutney's CI to match Tor's new test-network*We need to update the list of chutney networks in chutney's CI, to match the networks we added to Tor's `make test-network` and `make test-network-all` in #33334.We need to update the list of chutney networks in chutney's CI, to match the networks we added to Tor's `make test-network` and `make test-network-all` in #33334.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33378Require chutney node bootstrap before running verify2020-06-13T13:31:34ZteorRequire chutney node bootstrap before running verifyAs part of testing relay reachability self-tests in #33232, we need to make sure that all chutney nodes bootstrap.As part of testing relay reachability self-tests in #33232, we need to make sure that all chutney nodes bootstrap.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33379Make chutney wait for all relays in the consensus before verifying2020-06-13T13:31:35ZteorMake chutney wait for all relays in the consensus before verifyingAs part of #33232, we want chutney to check that all relays are in the consensus, then verify.As part of #33232, we want chutney to check that all relays are in the consensus, then verify.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33407Make chutney bridge authorities publish bridges in their networkstatus-bridges2020-06-13T15:51:43ZteorMake chutney bridge authorities publish bridges in their networkstatus-bridgesThis issue depends on the tor bridge descriptor upload fix in #33582, or robust reachability self-tests in #33222.
Chutney bridge authorities don't have any bridges in their networkstatus-bridges. That's a problem, because we want to ch...This issue depends on the tor bridge descriptor upload fix in #33582, or robust reachability self-tests in #33222.
Chutney bridge authorities don't have any bridges in their networkstatus-bridges. That's a problem, because we want to check networkstatus-bridges for the reachability checks in #33232.
Once bridges upload their descriptors, we can make chutney check the bridge authority's networkstatus-bridges for bridge nicknames (this ticket).
We can only do the networkstatus-bridges check on tor versions with the #33222 or #33582 fixes. So we'll need to check for:
* the next tor 0.4.4-alpha version, or
* an environmental variable that enables these tests.
We don't have to do these fixes, because it should be enough to test relay reachability. But we would risk breaking bridge reachability tests, and not knowing about it until after a release.Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33428Make chutney check for relay microdescriptors before verifying2020-06-13T13:31:37ZteorMake chutney check for relay microdescriptors before verifyingIn #33232, we make chutney check for relay and bridge descriptors, and relays in the consensus, before verifying. But we don't check for relay microdescriptors.
The easiest way to check for relay microdescriptors, is using each relay's ...In #33232, we make chutney check for relay and bridge descriptors, and relays in the consensus, before verifying. But we don't check for relay microdescriptors.
The easiest way to check for relay microdescriptors, is using each relay's ed25519 key. In #30642, we will add an `ed25519-identity` file to each relay's data directory. Chutney can read the contents of this file, and then search for it in `cached-microdescriptors` and `cached-microdescriptors.new`.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33581Restore bridge networkstatus checks in chutney2020-06-13T13:31:39ZteorRestore bridge networkstatus checks in chutneyThis issue depends on the tor bridge descriptor upload fix in #33582, robust reachability self-tests in #33222, or bridge log checks in #34037.
In chutney networks, there's a race condition when bridges try to publish their descriptor ...This issue depends on the tor bridge descriptor upload fix in #33582, robust reachability self-tests in #33222, or bridge log checks in #34037.
In chutney networks, there's a race condition when bridges try to publish their descriptor to the bridge authority:
* bridges try to publish their descriptors before bootstrapping
* but bridges can't publish their descriptors, because they don't have enough directory info to build a circuit to the bridge authority
Also, bridges do not retry publishing their descriptor immediately after they bootstrap.
We can only do the networkstatus-bridges check on tor versions with the #33222 or #33582 fixes. So we'll need to check for:
* the next tor 0.4.4-alpha version, or
* an environmental variable that enables these tests.
We don't have to do these fixes, because it should be enough to test relay reachability. But we would risk breaking bridge reachability tests, and not knowing about it until after a release.
Also, the chutney workarounds seem to cause weird race conditions, which are time-consuming to diagnose and fix.https://gitlab.torproject.org/legacy/trac/-/issues/33583Stop setting AssumeReachable on chutney relays and bridges2020-06-13T15:53:49ZteorStop setting AssumeReachable on chutney relays and bridgesWe need to stop setting AssumeReachable on relays and bridges in chutney networks, so they do reachability self-tests.
We should continue to set AssumeReachable on authorities (including the bridge authority), because it disables author...We need to stop setting AssumeReachable on relays and bridges in chutney networks, so they do reachability self-tests.
We should continue to set AssumeReachable on authorities (including the bridge authority), because it disables authority to relay reachability tests. (These tests are still performed on a half-hourly schedule, even in chutney networks. And they are out of scope for Sponsor 55.)
After we implement this change, we should also be able to implement #33581.teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33595Stop waiting for unchecked directory info2020-06-13T13:31:40ZteorStop waiting for unchecked directory infoOnce we've fixed #33428 (microdescs), #33582 (bridges), and #33609 (onion services), we might be able to remove WAIT_FOR_UNCHECKED_DIR_INFO in chutney's TorNet.py.
But we might still need to wait for:
* internal tor state changes, in re...Once we've fixed #33428 (microdescs), #33582 (bridges), and #33609 (onion services), we might be able to remove WAIT_FOR_UNCHECKED_DIR_INFO in chutney's TorNet.py.
But we might still need to wait for:
* internal tor state changes, in response to downloading descriptors.
So we will need to make this change, then test it on Tor 0.3.5 and master.https://gitlab.torproject.org/legacy/trac/-/issues/33596Fix or disable mixed+hs-v2 for Tor 0.3.52020-06-13T13:31:41ZteorFix or disable mixed+hs-v2 for Tor 0.3.5In #33379, we made some changes that broke mixed+hs-v2 on Tor 0.3.5. Strangely, hs-v2-min works fine.
It looks like it is caused by some microdescriptor download bugs that we fixed in 0.4.0 or 0.4.1.
When we finish the rest of #33232, ...In #33379, we made some changes that broke mixed+hs-v2 on Tor 0.3.5. Strangely, hs-v2-min works fine.
It looks like it is caused by some microdescriptor download bugs that we fixed in 0.4.0 or 0.4.1.
When we finish the rest of #33232, or fix #33428, we should either:
* fix this issue
* require tor or tor-stable to be 0.4.1 or later in mixed+hs-v2Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33597Travis: Revert to basic-min for Tor 0.3.52020-06-13T13:31:42ZteorTravis: Revert to basic-min for Tor 0.3.5In #33379, we changed CI to use single-onion-v23 for 0.3.5, because some other chutney networks were unstable.
Once #33428 or #33583 are fixed, we should be able to revert to basic-min.In #33379, we changed CI to use single-onion-v23 for 0.3.5, because some other chutney networks were unstable.
Once #33428 or #33583 are fixed, we should be able to revert to basic-min.https://gitlab.torproject.org/legacy/trac/-/issues/33598chutney does not fail on some SOCKS errors2020-06-13T13:31:42Zteorchutney does not fail on some SOCKS errorsWhen tor can't make a connection, and sends back a SOCKS error, chutney keeps on rapidly sending SOCKS requests. Instead, chutney should fail.
I think we introduced this bug when we started using asyncore.
I have worked around the bug...When tor can't make a connection, and sends back a SOCKS error, chutney keeps on rapidly sending SOCKS requests. Instead, chutney should fail.
I think we introduced this bug when we started using asyncore.
I have worked around the bug using a 5 second asyncore timeout, but we should come up with a permanent fix.
I think nickm might be able to help with this issue, because he wrote that code.cchttps://gitlab.torproject.org/legacy/trac/-/issues/33609Check that onion services have successfully posted descriptors before verifying2020-06-13T13:31:44ZteorCheck that onion services have successfully posted descriptors before verifyingBefore verifying, chutney checks that:
* each relay descriptor is cached at each node
* each relay is in a consensus, cached at each node
* each relay is in a microdesc consensus, cached at each node
* each bridge descriptor is cached at...Before verifying, chutney checks that:
* each relay descriptor is cached at each node
* each relay is in a consensus, cached at each node
* each relay is in a microdesc consensus, cached at each node
* each bridge descriptor is cached at each bridge client
We have other tickets for checking:
* microdescriptors
* cached bridge descriptors at the bridge authority
* the bridge networkstatus
That just leaves onion services.
Onion services are tricky, because they post to some HSDirs in the network, but not all. And those HSDirs don't cache the onion service descriptors in a file.
So here is one possible design for this feature:
* check each onion service log for a successful descriptor post to at least one HSDir
* check v2 and v3 onion services
* call it an extra 200% "bootstrap" stage (because it's a sender log check, not a receiver cached file check)
* require 200% bootstrap for onion servicescchttps://gitlab.torproject.org/legacy/trac/-/issues/33615Wait for at least 60 seconds for 0.3.5 and earlier to bootstrap2020-06-13T13:31:45ZteorWait for at least 60 seconds for 0.3.5 and earlier to bootstrapThe basic-min network is still a bit unstable, but only for 0.3.5. (And it's much more unrealiable on macOS than Linux.)
It seems like 0.3.5 needs around 60 seconds to bootstrap and sync descriptors. Later versions have microdescriptor ...The basic-min network is still a bit unstable, but only for 0.3.5. (And it's much more unrealiable on macOS than Linux.)
It seems like 0.3.5 needs around 60 seconds to bootstrap and sync descriptors. Later versions have microdescriptor download fixes, and they can bootstrap and verify in just over 20 seconds on basic-min.
So let's make the minimum wait time 70 seconds (3 consensuses), if any nodes in the network are running 0.3.5 (or earlier).teorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33617Add a BandwidthStatistics option and consensus parameter2020-06-13T15:52:12ZteorAdd a BandwidthStatistics option and consensus parameterStage 1:
We propose adding a new BandwidthStatistics torrc option, which activates reporting of all the bandwidth statistics. At the moment, all statistics are controlled by ExtraInfoStatistics, but we propose using the new BandwidthSt...Stage 1:
We propose adding a new BandwidthStatistics torrc option, which activates reporting of all the bandwidth statistics. At the moment, all statistics are controlled by ExtraInfoStatistics, but we propose using the new BandwidthStatistics option specifically for the bandwidth statistics.
The default value of this option should be 1. (The existing bandwidth statistics are reported by default.)
The new option should have a manual page entry, and test_parseconf.sh tests in src/test/conf_examples.
Stage 2:
Add a BandwidthStatistics consensus paramter.
Change the default value of the BandwidthStatistics option to "auto". If the value is "auto", use the value of the consensus parameter. If the consensus parameter is not set, use "1".
Update the manual page to describe the new consensus parameter.
See the proposal:
https://gitweb.torproject.org/torspec.git/tree/proposals/313-relay-ipv6-stats.txt#n298Tor: 0.4.4.x-finalMrSquancheeMrSquancheehttps://gitlab.torproject.org/legacy/trac/-/issues/33618Add IPv6 Support to is_local_addr()2020-06-13T15:52:13ZTracAdd IPv6 Support to is_local_addr()We propose this optional change, to improve the accuracy of IPv6 address
detection from directory documents.
Directory servers use is_local_addr() to detect if the requesting tor
instance is on the same local network. If it is, the dire...We propose this optional change, to improve the accuracy of IPv6 address
detection from directory documents.
Directory servers use is_local_addr() to detect if the requesting tor
instance is on the same local network. If it is, the directory server does
not include the X-Your-Address-Is HTTP header in directory documents.
Currently, is_local_addr() checks for:
* an internal IPv4 or IPv6 address, or
* the same IPv4 /24 as the directory server.
We propose also checking for:
* the same IPv6 /48 as the directory server.
We choose /48 because it is typically the smallest network in the global
IPv6 routing tables, and it was previously the recommended per-customer
network block. (See [RFC 6177: IPv6 End Site Address Assignment].)
Tor currently uses:
* IPv4 /8 and IPv6 /16 for port summaries,
* IPv4 /16 and IPv6 /32 for path selection (avoiding relays in the same
network block).
Source: https://gitweb.torproject.org/torspec.git/tree/proposals/312-relay-auto-ipv6-addr.txt#n1099
**Trac**:
**Username**: kimimaroTor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33633Move extend and reachability code to the relay module2020-06-13T15:52:17ZteorMove extend and reachability code to the relay moduleMost of the extend and reachability code is already in the relay module.
But some code was left behind in src/core/or/circuitbuild.c.Most of the extend and reachability code is already in the relay module.
But some code was left behind in src/core/or/circuitbuild.c.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33675Search microdescriptor files for relay ed25519 keys2020-06-13T13:31:47ZteorSearch microdescriptor files for relay ed25519 keysYour code in #33428 needs to pass your local "make test-network-all", before you start this ticket.
We need to enable searching for ed25519 keys in relay microdescriptor files.
There are instructions and a draft search pattern here:
ht...Your code in #33428 needs to pass your local "make test-network-all", before you start this ticket.
We need to enable searching for ed25519 keys in relay microdescriptor files.
There are instructions and a draft search pattern here:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L1325
Please open a new pull request for this ticket. Your branch should be based on the final version of #33428.
Before you push new changes to your pull request, your chutney code should pass:
* "make test-network-all" on tor master
* "make test-network-all" on tor maint-0.3.5
You can build a tor branch using these commands:
```
cd tor
git checkout -b <branch>
make
```
Where <branch> is master or maint-0.3.5https://gitlab.torproject.org/legacy/trac/-/issues/33676Stop waiting a set time for microdescriptors2020-06-13T13:31:48ZteorStop waiting a set time for microdescriptorsYour code in #33675 needs to pass your local "make test-network-all", on master and maint-0.3.5, before you start this ticket.
When we make chutney check for microdescriptors, we can stop waiting a set time for microdescriptors to downl...Your code in #33675 needs to pass your local "make test-network-all", on master and maint-0.3.5, before you start this ticket.
When we make chutney check for microdescriptors, we can stop waiting a set time for microdescriptors to download.
Please read these instructions carefully. Don't leave out any steps!
First, set MIN_START_TIME_LEGACY and NODE_WAIT_FOR_UNCHECKED_DIR_INFO to 0:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L926
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L952
Then check that `make test-network-all` still passes on:
* Tor master, and
* Tor maint-0.3.5
You can build a different tor branch using the following commands:
```
git checkout <branch>
make
```
Check the new code on your own machine, and then submit a pull request to run Chutney's CI.
If all those tests pass, you can remove all the code that uses MIN_START_TIME_LEGACY and NODE_WAIT_FOR_UNCHECKED_DIR_INFO:
* remove NODE_WAIT_FOR_UNCHECKED_DIR_INFO
* replace NODE_WAIT_FOR_UNCHECKED_DIR_INFO with 0 in getUncheckedDirInfoWaitTime()
* remove getMinStartTime()
* remove all the constants getMinStartTime() uses
* remove all calls to getMinStartTime()
* remove all the variables, code, and comments that depend on getMinStartTime()
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2224
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2273
Then add a commit to your pull request with these changes, and check that they pass CI.
You can ignore macOS hangs ("red ! symbol") on Travis CI. It is a bit unstable right now.https://gitlab.torproject.org/legacy/trac/-/issues/33677Stop waiting a set time for onion service descriptors2020-06-13T13:31:48ZteorStop waiting a set time for onion service descriptorsAfter we make chutney check for onion service descriptors in #33609, we can stop waiting a set time for onion service descriptors to download.
First, set HS_WAIT_FOR_UNCHECKED_DIR_INFO to 0:
https://github.com/torproject/chutney/blob/ma...After we make chutney check for onion service descriptors in #33609, we can stop waiting a set time for onion service descriptors to download.
First, set HS_WAIT_FOR_UNCHECKED_DIR_INFO to 0:
https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L953
Then check that `make test-network-all` still passes on:
* Tor master,
* Tor maint-0.3.5
Then check that chutney's CI still passes.
If all those tests pass, we can remove all the code that uses HS_WAIT_FOR_UNCHECKED_DIR_INFO:
* remove HS_WAIT_FOR_UNCHECKED_DIR_INFO
* remove getUncheckedDirInfoWaitTime()
* remove all the variables, code, and comments that depend on getUncheckedDirInfoWaitTime()
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2226
* https://github.com/torproject/chutney/blob/master/lib/chutney/TorNet.py#L2266https://gitlab.torproject.org/legacy/trac/-/issues/33679Make sure every address function that takes for_listening supports IPv62020-06-13T15:52:28ZteorMake sure every address function that takes for_listening supports IPv6We need to make sure all of our basic address functions support IPv6.
For example, tor_addr_is_valid() only supports IPv4 for_listening.
We need to make this change before we create generic IPv6 listeners for proposal 312.We need to make sure all of our basic address functions support IPv6.
For example, tor_addr_is_valid() only supports IPv4 for_listening.
We need to make this change before we create generic IPv6 listeners for proposal 312.Tor: 0.4.4.x-finalMrSquancheeMrSquancheehttps://gitlab.torproject.org/legacy/trac/-/issues/33681Refactor using_default_dir_authorities() local address checks2020-06-13T15:52:29ZteorRefactor using_default_dir_authorities() local address checksCurrently, we check for IPv4 and IPv6 local addresses and using_default_dir_authorities() in two different places:
* IPv4: resolve_my_address()
* IPv6: router_get_advertised_ipv6_or_ap()
Once we've combined the code that checks IPv4 and...Currently, we check for IPv4 and IPv6 local addresses and using_default_dir_authorities() in two different places:
* IPv4: resolve_my_address()
* IPv6: router_get_advertised_ipv6_or_ap()
Once we've combined the code that checks IPv4 and IPv6 advertised ORPorts, we can do a single using_default_dir_authorities() check.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33768Make tor_inet_pton() handle bad addresses consistently on Windows2020-06-13T15:52:40ZteorMake tor_inet_pton() handle bad addresses consistently on Windowstor_inet_pton() handles bad addresses differently on Windows and Linux/macOS.
For example, the address: "2000::1a00::1000:fc098" (two "::") fails this test on Windows, but succeeds on Linux and macOS:
https://github.com/torproject/tor/p...tor_inet_pton() handles bad addresses differently on Windows and Linux/macOS.
For example, the address: "2000::1a00::1000:fc098" (two "::") fails this test on Windows, but succeeds on Linux and macOS:
https://github.com/torproject/tor/pull/1831/commits/05f4f93722d46c0e8f1d09b4dea4bf5d1743d93f#diff-048243cd6d9ed36dda0944181d8ec8abR1729
Let's fix this bug and backport it.
In general, we should make all the functions in this file behave identically:
* zero any out parameters at the start of the function
* zero any out parameters on failureTor: 0.4.4.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33788Check the return value of tor_inet_ntop() and tor_inet_ntoa()2020-06-13T15:52:44ZteorCheck the return value of tor_inet_ntop() and tor_inet_ntoa()The following functions don't check the return value of tor_inet_ntop() or tor_inet_ntoa():
IPv6, could be serious:
* evdns_callback(), multiple times
IPv4 only, unlikely to be a serious bug:
* tor_dup_ip()
* fmt_addr32()
* evdns_wildc...The following functions don't check the return value of tor_inet_ntop() or tor_inet_ntoa():
IPv6, could be serious:
* evdns_callback(), multiple times
IPv4 only, unlikely to be a serious bug:
* tor_dup_ip()
* fmt_addr32()
* evdns_wildcard_check_callback()
These functions should log a bug log using BUG() or tor_assert_nonfatal(), and return an error. (Or for the formatting functions, a sensible placeholder string.)
We will also need to make their callers check for the error.Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33789Before making changes, move router pick address functions to their own C file2020-06-13T15:52:45ZteorBefore making changes, move router pick address functions to their own C fileAs part of Sponsor 55, let's move these relay functions to their own C files:
In src/feature/relay/resolve_addr_relay.c
* router_pick_published_address()
In src/app/config/resolve_addr.c
* resolve_my_address()
* get_last_resolved_addr(...As part of Sponsor 55, let's move these relay functions to their own C files:
In src/feature/relay/resolve_addr_relay.c
* router_pick_published_address()
In src/app/config/resolve_addr.c
* resolve_my_address()
* get_last_resolved_addr()
* reset_last_resolved_addr()
* last_resolved_addr
* is_local_addr()
* is_local_addr_impl()
These are relay-level now, but I think they should become client-level:
* router_new_address_suggestion()
* router_guess_address_from_dir_headers()
We should also move any related functions.
And write unit tests :-)Tor: 0.4.4.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33796socks: Prefer IPv6 by default on SOCKS port broke torsocks2020-06-13T15:52:46ZDavid Gouletdgoulet@torproject.orgsocks: Prefer IPv6 by default on SOCKS port broke torsocksIn commit `bf2a399fc0d90df76e091fa3259f7c1b8fb87781` we made all SocksPort to prefer IPv6 which means that any DNS resolution answer will pick IPv6, if exists, over IPv4.
For torsocks, this is a problem because by default it tries to re...In commit `bf2a399fc0d90df76e091fa3259f7c1b8fb87781` we made all SocksPort to prefer IPv6 which means that any DNS resolution answer will pick IPv6, if exists, over IPv4.
For torsocks, this is a problem because by default it tries to resolve an IPv4. This can be fixed _except_ when the libc call (ex: `getaddrinfo()`) is specifically requesting an IPv4 (`hints.ai_family = AF_INET`).
There is currently no way for torsocks to ask tor, via the SocksPort, a specific INET family and thus torsocks receives back the IPv6 and can't handle it because the application was expecting an IPv4.
So this example fails often as we have more and more Exits are able to resolve IPv6:
```
wget -4 some.url
```
And still many applications by default will request an IPv4 because they don't handle IPv6.
Bottom line is that torsocks use case is broken for when an application is specifically requesting an INET family...
As a reminder, torsocks can _not_ pass the hostname directly in the SOCKS connection because it hijacks libc calls and thus flow can only be 1) DNS resolution, 2) `connect()` with an IP address.
I'm not sure what to do here... I think the ideal scenario would be to have a way for our "resolve" SOCKS extension to specify an address family value.
For instance, the `F0` (`RESOLVE` command) would be "return whatever" and then we could extend to have `RESOLVE4` and `RESOLVE6`..... HACK-ish but those extensions are already a hack so...
(That prefer IPv6 change went in 043)Tor: 0.4.4.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33804Defer "PreferIPv6 by default" to 0.4.42020-06-13T15:52:47ZteorDefer "PreferIPv6 by default" to 0.4.4In #32637, we made clients default to "PreferIPv6" on SOCKSPorts (and other client ports).
But in #33796, we discovered that this change broke torsocks.
We want to defer this change to 0.4.4, so tor can implement IPv4-only and IPv6-onl...In #32637, we made clients default to "PreferIPv6" on SOCKSPorts (and other client ports).
But in #33796, we discovered that this change broke torsocks.
We want to defer this change to 0.4.4, so tor can implement IPv4-only and IPv6-only resolves for torsocks.
Since #32637, we have also merged:
* #33608, which removes a forced PreferIPv6 for non-SOCKSPorts
* #32994, which puts the PreferIPv6 default in port_parse_config(), rather than in 2 different places in the code
So it should be very easy to change the default for PreferIPv6.Tor: 0.4.3.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33812Add unit tests for bandwidth statistics functions2020-06-13T15:52:48ZMrSquancheeAdd unit tests for bandwidth statistics functionsWe currently don't have tests for bandwidth statistics
reporting code.
We need to write tests for the following functions.
1. commit_max()
2. advance_obs()
3. add_obs()
4. rep_hist_bandwidth_assess()
5. rep_hist_fill_bandwidth_history()
...We currently don't have tests for bandwidth statistics
reporting code.
We need to write tests for the following functions.
1. commit_max()
2. advance_obs()
3. add_obs()
4. rep_hist_bandwidth_assess()
5. rep_hist_fill_bandwidth_history()
6. rep_hist_get_bandwidth_lines()
This will be useful for #33617Tor: unspecifiedMrSquancheeMrSquancheehttps://gitlab.torproject.org/legacy/trac/-/issues/33816Fill in missing IPv6 addresses in extend cells2020-06-13T15:52:49ZteorFill in missing IPv6 addresses in extend cellsWhen an extend cell is missing an IPv6 address, tor relays could use the IPv6 address from the consensus. (If there is one.)
Similarly, if the client only supplied an IPv6 address, the relay could add an IPv4 address.
Relays already ad...When an extend cell is missing an IPv6 address, tor relays could use the IPv6 address from the consensus. (If there is one.)
Similarly, if the client only supplied an IPv6 address, the relay could add an IPv4 address.
Relays already add ed25519 keys during an extend, when the client only supplies the RSA fingerprint.
This change helps obfuscate:
* whether clients know the IPv6 addresses of relays,
* which clients implement sending IPv6 addresses in extends, and
* which clients are configured to send IPv6 addresses in extends.
This change also helps with reachability, if a relay has recently gained an IPv6 ORPort, or its IPv4 ORPort is unreliable.
It has a minor impact on testing:
* increases the number of IPv6 extends, but
* decreases the number of IPv4-only extends.
This change can be made in circuit_extend():
https://github.com/torproject/tor/pull/1801/files#diff-84b529c5e46d955c02d683463cd3317bR230
By calling a function that works like circuit_extend_add_ed25519_helper(), but adds IP addresses instead:
https://github.com/torproject/tor/pull/1801/files#diff-84b529c5e46d955c02d683463cd3317bR77Tor: 0.4.5.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/33817Perform Basic Relay IPv6 Extends2020-06-13T15:52:51ZteorPerform Basic Relay IPv6 ExtendsCurrently, tor checks that extend cells have IPv4 addresses in:
[x] some functions in circuitbuild_relay.c (a new file introduced by #33633)
[x] channel_get_for_extend() in channel.c
[x] check_extend_cell() in onion.c
[x] extend_cell_fr...Currently, tor checks that extend cells have IPv4 addresses in:
[x] some functions in circuitbuild_relay.c (a new file introduced by #33633)
[x] channel_get_for_extend() in channel.c
[x] check_extend_cell() in onion.c
[x] extend_cell_from_extend2_cell_body() in onion.c
[?] and possibly other functions.
We want to allow IPv6 in all these places.
And when we have an IPv4 and an IPv6 address, we want to choose between them at random.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33818Add options for clients and relays to enable IPv6 extends2020-06-13T15:52:53ZteorAdd options for clients and relays to enable IPv6 extendsTo help with testing and future network upgrades, we want to add options that:
* make clients and relays send IPv6 addresses in extend cells, and
* make relays perform extends over IPv6.
We also want to refactor the changes from #33817,...To help with testing and future network upgrades, we want to add options that:
* make clients and relays send IPv6 addresses in extend cells, and
* make relays perform extends over IPv6.
We also want to refactor the changes from #33817, so that we perform all the IPv6 mode checks in the same place. We want to modify tor's behaviour based on:
* tor's configuration
* including consensus parameters
* the reachability of a relay's own IPv6 ORPort, and
* any other relevant factors.
The functions that need to be refactored should all be in circuitbuild_relay.c.
We don't need to refactor or add IPv6 mode checks to the changes in #33899 or #33901. That includes these functions:
* connection_or_check_canonicity()
* channel_tls_process_netinfo_cell()
* channel_get_for_extend()
* check_extend_cell()
* extend_cell_from_extend2_cell_body()
But we might want to change how we *call* these functions from other code.
**General Options**
ExtendByIPv6ORPort (like ExtendByEd25519ID):
If this option is set to 1, Tor tries to include a relay’s IPv6 ORPort when telling the preceding relay in a circuit to extend to it. When IPv6 extends are enabled, relays and bridges use them to perform IPv6 reachability self-tests. Once these tests succeed, they publish their descriptors. (See AssumeReachable for more details.)
If this option is set to 0, Tor never includes an IPv6 ORPort when extending circuits. Tor relays and bridges are unable to perform IPv6 reachability self-tests.
If the option is set to "auto", Tor obeys a parameter in the consensus document. If the consensus parameter is not set:
* relays include IPv6 ORPorts in extend cells,
* bridges only include IPv6 ORPorts in the final extend in reachability self-test circuits,
* clients do not include IPv6 ORPorts in extend cells.
(Default: auto)
**Note:** there's a design tradeoff here:
* Gradual IPv6 upgrade:
* we can have 3 different behaviours:
* relay always sends IPv6
* bridge sends IPv6 on reachability
* client never sends IPv6
* and do a gradual IPv6 upgrade:
* stage 1: relay all, bridge reachability
* stage 2: relay, bridge, client all; or
* Big Bang IPv6 upgrade:
* we can have 2 different behaviours:
* relay, bridge sends IPv6 on reachability
* client never sends IPv6
* and do a big bang IPv6 upgrade:
* stage 1: relay and bridge reachability,
* stage 2: relay, bridge, client all
I think the gradual upgrade is less risky for the network, and the complexity of the extra code, behaviour, and testing is manageable.
**Relay and Bridge Options**
ExtendAllowIPv6Addresses (like ExtendAllowPrivateAddresses):
Relays and bridges only.
When this option is set to 1, Tor relays and bridges allow EXTEND requests to IPv6 addresses. When IPv6 extends are enabled in their own configurations, relays and bridges assume that other relays in the network will also allow IPv6 extends. So they perform IPv6 reachability self-tests, before publishing their descriptors. (See AssumeReachable for more details.)
This option does not apply to clients, or direct OR connections initiated by relays or bridges. Use ClientUseIPv6 and ClientPreferIPv6ORPort to enable direct IPv6 OR connections.
If this option is set to 0, Tor relays and bridges do not connect to IPv6 ORPorts when extending circuits. When IPv6 extends are disabled, relays and bridges assume that other relays in the network may also refuse IPv6 extends. So they continue to perform IPv6 reachability self-tests, but consider them unreliable. They publish their descriptors, regardless of the outcome of the tests.
If the option is set to "auto", Tor obeys a parameter in the consensus document. If the consensus parameter is not set:
* relays allow IPv6 extends,
* bridges do not allow IPv6 extends, and
* relays and bridges perform IPv6 reachability self-tests, before publishing their descriptors.
(Default: auto)
**Proposal Notes**
The design and option names are changed from section 4.4.4 of proposal 311, for the following reasons:
* consistent, safer design for ExtendByIPv6ORPort:
* relay reachability and other cells can't be distinguished
* relays include IPv6 first, because they don't require anonymity
* clients wait for deployment, before including IPv6 on a flag day
* bridges match clients
* ExtendAllowIPv6Addresses can wait for clients to learn IPv6
* consistency with existing options
* AssumeIPv6Reachable is no longer required, it is obsoleted by:
* AssumeReachable 1 (skip IPv4 and IPv6 reachability)
* ExtendByIPv6ORPort 0 (skip IPv6 reachability)
* ExtendAllowIPv6Addresses 0 (only log IPv6 reachability)
See:
https://github.com/torproject/torspec/blob/master/proposals/311-relay-ipv6-reachability.txt#L535Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33819Make clients and bridges send IPv6 extends by default in Tor 0.4.52020-06-13T15:52:54ZteorMake clients and bridges send IPv6 extends by default in Tor 0.4.5If #33818 is included in Tor 0.4.4, the relays will send IPv6 extends by default. But clients and bridges won't, for anonymity reasons.
By the time 0.4.5 is released, a large number of clients and bridges will be on 0.4.4. So it will be...If #33818 is included in Tor 0.4.4, the relays will send IPv6 extends by default. But clients and bridges won't, for anonymity reasons.
By the time 0.4.5 is released, a large number of clients and bridges will be on 0.4.4. So it will be safe for clients and bridge to send IPv6 extends.
Here's what we need to do to change the default:
* set `ExtendByIPv6ORPort=1` in the consensus, and
* set `ExtendByIPv6ORPort 1` as the default in Tor.
To avoid confusion, clients and bridges will send IPv6 extends, but they won't initiate outbound IPv6 connections to relays. For more details, see #33818 and ExtendAllowIPv6Addresses.Tor: 0.4.5.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33825Make Environ handle "in" and "get()" like a dict2020-06-13T13:31:49ZteorMake Environ handle "in" and "get()" like a dictSome standard Python dict code doesn't work on chutney's Environ class:
```
is_in_env = 'foo' in self._env
value_or_none = self._env.get('foo')
```
"in" should return a boolean, and "get()" should return the value (or None).
But instea...Some standard Python dict code doesn't work on chutney's Environ class:
```
is_in_env = 'foo' in self._env
value_or_none = self._env.get('foo')
```
"in" should return a boolean, and "get()" should return the value (or None).
But instead, when the key is missing, sometimes they throw a KeyError. (It seems to happen in certain contexts, but not others.)
We should work out if Environ is missing some of the required dict implementation functions. Or if there is some issue with Environ's lookup code.
Then we should implement unit tests, to make sure we don't break Environ in future.https://gitlab.torproject.org/legacy/trac/-/issues/33826Add a testing-only option that turns off IPv4 extends2020-06-13T15:52:55ZteorAdd a testing-only option that turns off IPv4 extendsTo test that relays perform IPv6 extends correctly, we should turn off all IPv4 extends in a test network.
The option should be called ExtendAllowIPv4Addresses, and it should be documented and implemented like ExtendAllowIPv6Addresses i...To test that relays perform IPv6 extends correctly, we should turn off all IPv4 extends in a test network.
The option should be called ExtendAllowIPv4Addresses, and it should be documented and implemented like ExtendAllowIPv6Addresses in #33818.
We might also need an ExtendByIPv4ORPort option, similar to ExtendByIPv6ORPort.Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33860Finish test_onionskin_answer()2020-06-13T15:53:00ZteorFinish test_onionskin_answer()In #33633, we finished unit tests for test_extend() and helpers.
But we didn't have time to finish test_onionskin_answer().
Let's try to test each of the cases of each `if` statement in onionskin_answer(). It's ok to mock the functions...In #33633, we finished unit tests for test_extend() and helpers.
But we didn't have time to finish test_onionskin_answer().
Let's try to test each of the cases of each `if` statement in onionskin_answer(). It's ok to mock the functions that are called by onionskin_answer().Tor: unspecifiedNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33880Confusing "Your relay has a very large number of connections to other relays"...2020-06-13T15:53:02ZRoger DingledineConfusing "Your relay has a very large number of connections to other relays" relay messageA new relay operator reports this complaint in their logs, showing up hourly:
```
Your relay has a very large number of connections to other relays. Is your
outbound address the same as your relay address? Found 13 connections to 8
relay...A new relay operator reports this complaint in their logs, showing up hourly:
```
Your relay has a very large number of connections to other relays. Is your
outbound address the same as your relay address? Found 13 connections to 8
relays. Found 13 current canonical connections, in 0 of which we were a
non-canonical peer. 5 relays had more than 1 connection, 0 had more than 2, and
0 had more than 4 connections.
```
I checked, and their outbound address was the same as their relay address.
Upon investigation, it looks like the redundant connections are to directory authorities.
My theory is that the change from #17592 (which went into 0.3.1.1-alpha, commit d5a151a0) is responsible: while before that canonical relay-to-relay connections would expire after either side first reached its randomized "15 to 22.5 minute" timeout, now they expire after either side reaches its "45 to 75 minute" timeout. And since directory authorities test reachability every 1280 seconds (around 21.3 minutes), that means it is expected that most relays will have duplicate canonical connections with directory authorities.
Possible fixes:
(A) Change the notice-level log to make it clearer that it's not scary, or at least it's not actionable. Maybe that means making it info-level so nobody will see it. Probably not the best option, assuming there *are* cases where we do want relay operators to hear it.
(B) In channel_check_for_duplicates(), change `MIN_RELAY_CONNECTIONS_TO_WARN 5` to a high enough number that even if we have 2 canonical conns per authority, plus a bit more, the log message still doesn't trigger.
(C) In channel_check_for_duplicates(), skip over connections to directory authorities in some way, since we know they will be special.
(D) Make connections to or from directory authorities expire quicker, on the theory that they don't really need the same level of padding protection as other connections.
(E) Your idea here?
I'd be fine with any of B,C,D. Whichever one can be done with an easy, short, and non-invasive patch is my favorite. Maybe that's "B, raise it to 30"? That would make the message trigger when we have connections to more than 30 relays and also we have more than 45 connections open. Or we could pick the more conservative "raise it to 40", on the theory that small numbers are more likely to have edge cases and less indicative of major network problems anyway.
And while we're at it, it might be smart to say in the log message what action we want the relay operator take, e.g. "Please report:".Tor: unspecifiedNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33898Stop modifying addr on connections, and delete real_addr2020-06-13T15:53:06ZteorStop modifying addr on connections, and delete real_addrIn connection_or_check_canonicity(), we overwrite conn.addr with the canonical address of the relay in the consensus. That makes accurate logging impossible.
And so we also need channel.real_addr, to store the actual address of the remo...In connection_or_check_canonicity(), we overwrite conn.addr with the canonical address of the relay in the consensus. That makes accurate logging impossible.
And so we also need channel.real_addr, to store the actual address of the remote peer. And for some reason, we also have conn.address, a string copy of the peer's canonical address and port.
See:
https://github.com/torproject/tor/blob/7f9eaec538b7d01e0d1b130dc4cf2ec634252d46/src/core/or/connection_or.c#L920
This is... a mess.
Here's the high-level interface I'd like to see:
* use a function to format a connection or channel addresses for loogging
* use exactly as many address and port variables as needed in connection and channel (no extras!)
* qualify each address and port variable's name with its purpose
For example, here's one possible design:
* delete addr, port, address, and real_addr
* add remote_ap, a tor_addr_port_t that is the remote address and port of the TCP connection to the remote peer
* implement connection_describe(), which:
* formats remote_ap,
* formats is_canonical (and any other useful info), and
* calls node_describe() to format the canonical IPv4 and IPv6 addresses and ports of the remote peer.
We may also need separate fields for reverse proxied addresses, see the comment at:
https://github.com/torproject/tor/blob/7517e1b5d31aada1f594c2594737a231d9d8e116/src/core/or/channeltls.c#L1339
If we need separate variables or functions for channels, we can use a similar design. (But ideally, re-using as many functions and variables as possible.)
This is important for Sponsor 55: getting accurate connection information will make diagnosing bugs much easier.Tor: 0.4.5.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/33899Allow IPv6 addresses to be canonical2020-06-13T15:53:06ZteorAllow IPv6 addresses to be canonicalIn #17604, we allowed IPv6 connections to set is_canonical_to_peer. But we did not let them set is_canonical.
So channel_tls_process_netinfo_cell() handles IPv6 addresses, but connection_or_check_canonicity() does not.
This inconsisten...In #17604, we allowed IPv6 connections to set is_canonical_to_peer. But we did not let them set is_canonical.
So channel_tls_process_netinfo_cell() handles IPv6 addresses, but connection_or_check_canonicity() does not.
This inconsistency could be one cause of the logging issues in #24841.
We should allow IPv6 connections to be canonical. We should also do some cleanup of the related connection address logging in #33898.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33900Check for invalid zero IPv4 addresses and ports in extend cells2020-06-13T15:53:07ZteorCheck for invalid zero IPv4 addresses and ports in extend cellsWhen we send and parse extend cells, we check that their IPv4 address field is not AF_UNSPEC.
But we should also check for zero IPv4 addresses and zero ports. (Which are both invalid.)
Code and points in #33817, I just needed a separat...When we send and parse extend cells, we check that their IPv4 address field is not AF_UNSPEC.
But we should also check for zero IPv4 addresses and zero ports. (Which are both invalid.)
Code and points in #33817, I just needed a separate bug number.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33901Allow IPv6 extend cells2020-06-13T15:53:08ZteorAllow IPv6 extend cellsAs part of #33817, we want to allow IPv6 extend cells.
We want clients and relays to send:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cells
We want relays to parse:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cellsAs part of #33817, we want to allow IPv6 extend cells.
We want clients and relays to send:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cells
We want relays to parse:
* dual-stack EXTEND2 cells
* IPv6-only EXTEND2 cellsTor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33917Make IF_BUG_ONCE() support ALL_BUGS_ARE_FATAL2020-06-13T15:53:09ZteorMake IF_BUG_ONCE() support ALL_BUGS_ARE_FATAL... and DISABLE_ASSERTS_IN_UNIT_TESTS.
It's the only assertion/bug macro that doesn't support these debugging modes. Let's fix that.... and DISABLE_ASSERTS_IN_UNIT_TESTS.
It's the only assertion/bug macro that doesn't support these debugging modes. Let's fix that.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33918Stop truncating IPv6 addresses in channel logs2020-06-13T15:53:10ZteorStop truncating IPv6 addresses in channel logschannel_tls_get_remote_desc_method() formats IP addresses and ports, but its buffer is only 32 characters. IPv6 addresses can be up to 41 characters long, and the port is an extra 6 characters.channel_tls_get_remote_desc_method() formats IP addresses and ports, but its buffer is only 32 characters. IPv6 addresses can be up to 41 characters long, and the port is an extra 6 characters.Tor: 0.4.4.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/33919Write unit tests for channel_matches_target_addr_for_extend()2020-06-13T15:53:10ZteorWrite unit tests for channel_matches_target_addr_for_extend()In #33817, we modify channel_matches_target_addr_for_extend() to take an IPv4 and an IPv6 address.
We also modify channel_get_for_extend() to take an IPv4 and an IPv6 address. It has existing tests, but they ignore IP addresses. (And mo...In #33817, we modify channel_matches_target_addr_for_extend() to take an IPv4 and an IPv6 address.
We also modify channel_get_for_extend() to take an IPv4 and an IPv6 address. It has existing tests, but they ignore IP addresses. (And mock the matches_target() method, which is called by channel_matches_target_addr_for_extend().)
It would be useful to have tests that check that a channel matches, if its IP address matches the IPv4 or IPv6 address. But it's not urgent. (And the actual changes to the function are pretty trivial.)
The setup for these tasks is a bit tricky, we need to set up a channel_tls_t and an associated connection_t with a real_addr. (Note that #33898 may change the name of real_addr.)
This task is not urgent or important. Anyone can do this task.Tor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/34037Make chutney check tor's logs for reachability self-test success2020-06-13T15:52:08ZteorMake chutney check tor's logs for reachability self-test successThis ticket is an alternative to #33582 or #33222.
Instead of fixing bridge descriptor uploads, we can check bridge logs to make sure that reachability self-tests have succeeded.
For consistency, we should also do the same checks for r...This ticket is an alternative to #33582 or #33222.
Instead of fixing bridge descriptor uploads, we can check bridge logs to make sure that reachability self-tests have succeeded.
For consistency, we should also do the same checks for relays.
We can only do these tests on authorities, relays, and bridges that are configured with `AssumeReachable 0`. Chutney's current defaults are:
* directory authorities: 1
* bridge authorities: 1
* relays: 0
* bridges: 0
* clients: clients never perform reachability self-tests
Some custom chutney networks may set `AssumeReachable 1` for relays and bridges. So we should make it easy for them to disable these checks.https://gitlab.torproject.org/legacy/trac/-/issues/34065Make routerset_contains_router() support IPv62020-06-13T15:53:18ZteorMake routerset_contains_router() support IPv6In router_should_check_reachability(), we test to see if the current relay is in ExcludeNodes.
But ExcludeNodes doesn't support IPv6, so if a user excludes our IPv6 address, we'll just ignore that setting.
This is not a required task.
...In router_should_check_reachability(), we test to see if the current relay is in ExcludeNodes.
But ExcludeNodes doesn't support IPv6, so if a user excludes our IPv6 address, we'll just ignore that setting.
This is not a required task.
But we should probably be explicit about ignoring IPv6 in the man page.Tor: 0.4.4.x-finalNeel Chauhanneel@neelc.orgNeel Chauhanneel@neelc.orghttps://gitlab.torproject.org/legacy/trac/-/issues/34067Separate tor's IPv4 and IPv6 reachability flags2020-06-13T15:53:19ZteorSeparate tor's IPv4 and IPv6 reachability flagsIn #33222, I make tor build circuits to IPv4 and IPv6 ORPorts.
But we also need to detect IPv4 and IPv6 reachability separately.In #33222, I make tor build circuits to IPv4 and IPv6 ORPorts.
But we also need to detect IPv4 and IPv6 reachability separately.Tor: 0.4.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/34068Decide how to handle control port events for IPv6 reachability self-tests2020-06-13T15:53:19ZteorDecide how to handle control port events for IPv6 reachability self-testsThe control spec has two reachability self-test events.
Here is how they are specified:
```
CHECKING_REACHABILITY
"ORADDRESS=IP:port"
"DIRADDRESS=IP:port"
We're going to start testing the reachability of our extern...The control spec has two reachability self-test events.
Here is how they are specified:
```
CHECKING_REACHABILITY
"ORADDRESS=IP:port"
"DIRADDRESS=IP:port"
We're going to start testing the reachability of our external OR port
or directory port.
REACHABILITY_SUCCEEDED
"ORADDRESS=IP:port"
"DIRADDRESS=IP:port"
We successfully verified the reachability of our external OR port or
directory port (depending on which of ORADDRESS or DIRADDRESS is
given.)
```
And here is what tor actually sends:
```
CHECKING_REACHABILITY ORADDRESS=IPv4:port
CHECKING_REACHABILITY DIRADDRESS=IPv4:port
REACHABILITY_SUCCEEDED ORADDRESS=IPv4:port
REACHABILITY_SUCCEEDED DIRADDRESS=IPv4:port
```
When we add IPv6 reachability events, we could break some (buggy) control parsers with:
```
CHECKING_REACHABILITY ORADDRESS=[IPv6]:port
REACHABILITY_SUCCEEDED ORADDRESS=[IPv6]:port
```
How should we handle this change?Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/34069Allow extend_info to contain both IPv4 and IPv6 ORPorts2020-06-13T15:53:20ZteorAllow extend_info to contain both IPv4 and IPv6 ORPortsTo send dual-stack EXTEND2 cells in #33222, we need to have IPv4 and IPv6 addresses in extend_info_t.To send dual-stack EXTEND2 cells in #33222, we need to have IPv4 and IPv6 addresses in extend_info_t.Tor: 0.4.5.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/34137Make sure inform_testing_reachability() reports the correct ports2020-06-13T15:53:29ZteorMake sure inform_testing_reachability() reports the correct portsIn #33222, we added IPv6 ORPort support to inform_testing_reachability(). But that function checks the ORPorts and DirPort itself, rather than logging the reachability checks that were actually launched.
We'd like to pass flags so that ...In #33222, we added IPv6 ORPort support to inform_testing_reachability(). But that function checks the ORPorts and DirPort itself, rather than logging the reachability checks that were actually launched.
We'd like to pass flags so that it logs the actual reachability tests that are being run. (Rather than re-checking the relay's own routerinfo, which may have changed since the most recent reachability checks were launched.)
But inform_testing_reachability() is called when:
* the first circuit finishes building, or
* tor is reconfigured, and some circuits have already finished building.
So we need to do a bit of a refactor.
The refactor should preserve this behaviour:
* don't log until after the first circuit has finished building (rather than logging as soon as we start building reachability circuits)
And introduce this new behaviour:
* log the ports that were tested recently, rather than the ports that are currently available.
As an alternative, we could move some of the logging into the functions that actually launch the checks. And elevate some of those logs to notice level. (Note that these checks can be launched from at least 4 different locations in tor's code.)Tor: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/34200Refactor tor's circuit path node selection checks2020-06-13T15:53:32ZteorRefactor tor's circuit path node selection checksIn #33222, we added an extra "can extend over IPv6" check to tor's circuit path node selection code.
To make sure it's applied consistently, I did a refactor of that code, so all those checks are in one function.In #33222, we added an extra "can extend over IPv6" check to tor's circuit path node selection code.
To make sure it's applied consistently, I did a refactor of that code, so all those checks are in one function.Tor: 0.4.5.x-finalteorteorhttps://gitlab.torproject.org/legacy/trac/-/issues/34412Additonal unit tests for functions related to ipv6 self-test2020-06-13T15:53:47ZNick MathewsonAdditonal unit tests for functions related to ipv6 self-testSee #33222 and #34200 for lists of functions that did not get unit tests as part of the #34200 merge.See #33222 and #34200 for lists of functions that did not get unit tests as part of the #34200 merge.Tor: 0.4.5.x-final