The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2020-06-27T14:25:52Zhttps://gitlab.torproject.org/tpo/network-health/metrics/website/-/issues/27000Stop providing pre-aggregated CSV files after September 152020-06-27T14:25:52ZKarsten LoesingStop providing pre-aggregated CSV files after September 15On July 31 we deployed an updated `stats.html` where we declared all pre-aggregated CSV files as deprecated. We said we'd remove those from the public interface after September 15. This should be just a trivial patch, so this ticket is m...On July 31 we deployed an updated `stats.html` where we declared all pre-aggregated CSV files as deprecated. We said we'd remove those from the public interface after September 15. This should be just a trivial patch, so this ticket is mostly here to remind us to do this on or shortly after September 15.
This is still part of legacy/trac#25383.Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/tpo/core/tor/-/issues/27378Stop providing the hs flag on chutney authorities2020-06-27T13:52:22ZteorStop providing the hs flag on chutney authoritiesThis flag has been ignored since well before 0.2.9.This flag has been ignored since well before 0.2.9.teorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/32954Stop public authorities specifying an internal address for their IPv6 ORPort2022-06-24T14:53:56ZteorStop public authorities specifying an internal address for their IPv6 ORPortWhen we implement `Address [IPv6]` in legacy/trac#30954, we should check that the supplied IPv6 address is a public address on Directory Authorities.
This matches the current `Address IPv4` check for authorities.When we implement `Address [IPv6]` in legacy/trac#30954, we should check that the supplied IPv6 address is a public address on Directory Authorities.
This matches the current `Address IPv4` check for authorities.https://gitlab.torproject.org/tpo/core/arti/-/issues/944Stop publishing unnecessary crates.2023-07-05T16:49:42ZNick MathewsonStop publishing unnecessary crates.At least `arti-bench` is useless to anybody besides us. We should stop publishing this and other useless/internal crates.At least `arti-bench` is useless to anybody besides us. We should stop publishing this and other useless/internal crates.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/core/tor/-/issues/8295Stop reading connection when marking it for close2020-06-27T14:04:42ZcypherpunksStop reading connection when marking it for closeconnection_handle_read_impl() can't handle marked for close connection. No purpose to eat CPU cycles to call conn_read_callback() if no data will read. _connection_mark_for_close() should to call connection_stop_reading()connection_handle_read_impl() can't handle marked for close connection. No purpose to eat CPU cycles to call conn_read_callback() if no data will read. _connection_mark_for_close() should to call connection_stop_reading()Tor: 0.2.4.x-finalhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/18314Stop recreating NoScript whitelists!2022-06-17T21:41:30ZcypherpunksStop recreating NoScript whitelists!Stop recreating NoScript whitelists! I removed everything from them including the changelog, but they appears again!Stop recreating NoScript whitelists! I removed everything from them including the changelog, but they appears again!https://gitlab.torproject.org/tpo/network-health/metrics/onionoo/-/issues/23796Stop rejecting field queries due to capitalisation2020-06-27T14:24:04ZteorStop rejecting field queries due to capitalisationOn my iOS device, when I type a field name in the Atlas search bar, it capitalises the first letter. For example: "Contact:teor".
We should stop rejecting queries like this as malformed.On my iOS device, when I type a field name in the Atlas search bar, it capitalises the first letter. For example: "Contact:teor".
We should stop rejecting queries like this as malformed.https://gitlab.torproject.org/tpo/core/tor/-/issues/20839Stop rejecting NS descriptors when microdescs are the default2020-06-27T13:57:42ZteorStop rejecting NS descriptors when microdescs are the defaultWhen we merged legacy/trac#6769, it exposed a bug in router_get_routerlist(), which checks full router descriptors against the usable consensus flavour. This check fails for all downloaded descriptors when the usable flavour is microdesc...When we merged legacy/trac#6769, it exposed a bug in router_get_routerlist(), which checks full router descriptors against the usable consensus flavour. This check fails for all downloaded descriptors when the usable flavour is microdescriptor.
Directory servers were somewhat immune to this issue, because they still put the descriptors that failed this check in old_routers. But this still might have caused subtle bugs down the track.Tor: 0.3.0.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/29570Stop relays publishing descriptors containing NoListen IPv6 ORPorts2020-07-29T03:08:55Zs7rStop relays publishing descriptors containing NoListen IPv6 ORPortsFor details:
https://lists.torproject.org/pipermail/tor-relays/2019-February/016988.html
and my reply:
https://lists.torproject.org/pipermail/tor-relays/2019-February/016991.html
This is one rare and strange setup of using IPv6 in a w...For details:
https://lists.torproject.org/pipermail/tor-relays/2019-February/016988.html
and my reply:
https://lists.torproject.org/pipermail/tor-relays/2019-February/016991.html
This is one rare and strange setup of using IPv6 in a way it is not intended, but we should still make sure that:
- if ORPort [IPv6:address::x]:port **NoListen** was set in torrc, and there is no following ORPort [IPv6:address::y]:port **NoAdvertise** or [::]:port **NoAdvertise** (as in use all available IPv6 addresses) is set, warn in the log and **do not build the descriptor using the NoListen address**, since the daemon is not listening on any address from the v6 class.
- check if the logic is applied for IPv4 also, even it's impossible to experience this in IPv4 since UnreachableIPv4 doesn't exist and can't possibly exist.
Otherwise we fill the descriptor with useless data and also have the directory authorities chase green horses.
I think we have this since forever, but not marking this as a backport given the rare cases when it can occur and the state of current IPv6 adoption.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/22700Stop relays using the Client*IPv6* options2020-07-28T03:40:49ZteorStop relays using the Client*IPv6* optionsSome operators add all the IPv6 options to their torrcs.
We should warn when they add client options on relays.Some operators add all the IPv6 options to their torrcs.
We should warn when they add client options on relays.Tor: unspecifiedhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27096Stop relying on $HOME being set in the unit tests2020-06-27T13:52:37ZteorStop relying on $HOME being set in the unit testsWhen I test Tor with $HOME unset, I see the following error:
```
options/validate__uname_for_server: [forking]
FAIL ../src/test/test_options.c:507: expected log to not contain entries Captured logs:
1. warn: "Couldn\'t find $HOME...When I test Tor with $HOME unset, I see the following error:
```
options/validate__uname_for_server: [forking]
FAIL ../src/test/test_options.c:507: expected log to not contain entries Captured logs:
1. warn: "Couldn\'t find $HOME environment variable while expanding \"~/.tor\"; defaulting to \"\".\n"
2. warn: "Default DataDirectory is \"~/.tor\". This expands to \"/.tor\", which is probably not what you want. Using \"/usr/local/var/tor\" instead\n"
[validate__uname_for_server FAILED]
```
This test has been around since 0.2.8.1-alpha.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/network-health/metrics/library/-/issues/21932Stop relying on the platform's default charset2020-06-27T14:23:38ZKarsten LoesingStop relying on the platform's default charsetWhile looking into the encoding issue of different Onionoo instances producing different contact string encodings (legacy/trac#15813), I tracked down this issue to metrics-lib's `ServerDescriptorImpl.java` class and its usage of `new Str...While looking into the encoding issue of different Onionoo instances producing different contact string encodings (legacy/trac#15813), I tracked down this issue to metrics-lib's `ServerDescriptorImpl.java` class and its usage of `new String(byte[])`.
The issue is that the constructor above uses "the platform's default charset". Turns out that the main Onionoo instance uses `US-ASCII` as default charset (`Charset.defaultCharset()`) and the mirror uses `UTF-8`. (Interestingly, the mirror only uses `UTF-8` for commands executed by cron and also uses `US-ASCII` for commands directly executed by my user, so the default would change depending on whether Onionoo's updater was started automatically after a reboot or started manually by the user; which made debugging just a bit more challenging!)
Long story short, we should not rely on the platform's default charset when converting bytes to strings or vice versa, but we should explicitly specify the charset we want! We just need to pick one.
Somewhat related I ran an analysis of character encodings in relay server descriptors two weeks ago. Here's what I found:
```
$ wget
https://collector.torproject.org/archive/relay-descriptors/server-descriptors/server-descriptors-2017-02.tar.xz
$ tar xf server-descriptors-2017-02.tar.xz
$ find server-descriptors-2017-02 -type f -exec file --mime {} \; > mimes
$ cut -d" " -f3 mimes | sort | uniq -c
68 charset=iso-8859-1
466900 charset=us-ascii
1145 charset=utf-8
```
I'd say let's just pretend that server descriptors are UTF-8 encoded. In this case, the following patch will resolve the issue for server descriptors:
```
diff --git a/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java b/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java
index 309cad4..2381378 100644
--- a/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java
+++ b/src/main/java/org/torproject/descriptor/impl/ServerDescriptorImpl.java
@@ -8,6 +8,7 @@ import org.torproject.descriptor.DescriptorParseException;
import org.torproject.descriptor.ServerDescriptor;
import java.io.UnsupportedEncodingException;
+import java.nio.charset.StandardCharsets;
import java.security.MessageDigest;
import java.security.NoSuchAlgorithmException;
import java.util.ArrayList;
@@ -56,8 +57,8 @@ public abstract class ServerDescriptorImpl extends DescriptorImpl
}
private void parseDescriptorBytes() throws DescriptorParseException {
- Scanner scanner = new Scanner(new String(this.rawDescriptorBytes))
- .useDelimiter("\n");
+ Scanner scanner = new Scanner(new String(this.rawDescriptorBytes,
+ StandardCharsets.UTF_8)).useDelimiter("\n");
String nextCrypto = "";
List<String> cryptoLines = null;
while (scanner.hasNext()) {
```
If this sounds like a reasonable plan, we should look into other places in the code where we use methods relying on the platform's default charset and explicitly specify a charset there, too.metrics-lib 2.0.0https://gitlab.torproject.org/tpo/network-health/sbws/-/issues/28103Stop removing results that are not away from some other X secs2020-06-27T13:41:35ZjugaStop removing results that are not away from some other X secsIn legacy/trac#27338 it was implemented to do not include relays in the bandwidth files that does not have at least 2 measurements that are X secs away from each other.
It was implemented removing any results that were not X secs away fr...In legacy/trac#27338 it was implemented to do not include relays in the bandwidth files that does not have at least 2 measurements that are X secs away from each other.
It was implemented removing any results that were not X secs away from each other, when all the results should be keep when there is at least one away from the others, as commented in legacy/trac#28061.sbws: 1.0.x-finaljugajugahttps://gitlab.torproject.org/tpo/core/tor/-/issues/30958Stop removing the ed25519 signature when the extra-info file is too big2020-06-27T13:49:56ZteorStop removing the ed25519 signature when the extra-info file is too bigIn legacy/trac#30956, I discovered that the ed25519 signature extra-info line is
split across two chunks.
If the extra-info file gets too big, tor removes one chunk at a time. So each chunk needs to be a complete line.
Edit: but in thi...In legacy/trac#30956, I discovered that the ed25519 signature extra-info line is
split across two chunks.
If the extra-info file gets too big, tor removes one chunk at a time. So each chunk needs to be a complete line.
Edit: but in this case, we should just stop removing the signatureTor: 0.4.2.x-finalteorteorhttps://gitlab.torproject.org/tpo/core/chutney/-/issues/29760Stop repeating chutney's defaults in comments2020-06-27T13:18:41ZteorStop repeating chutney's defaults in commentsWhen we repeat the defaults in comments and code, the comments get out of date,
Example:
```
# chutney starts verifying after START seconds, keeps on trying for BOOTSTRAP seconds,
# and then stops after STOP seconds (see the defaults be...When we repeat the defaults in comments and code, the comments get out of date,
Example:
```
# chutney starts verifying after START seconds, keeps on trying for BOOTSTRAP seconds,
# and then stops after STOP seconds (see the defaults below)
```
https://github.com/teor2345/chutney/pull/2/files#r264898496teorteorhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27402stop reporting "internal paths" during bootstrap2020-06-27T13:52:21ZTaylor Yustop reporting "internal paths" during bootstrap`bootstrap_status_to_string()` can report "internal paths" when the consensus actually contains exits, because it's conditional on `router_have_consensus_path()` which checks descriptors. In the cold cache case, there are likely no exis...`bootstrap_status_to_string()` can report "internal paths" when the consensus actually contains exits, because it's conditional on `router_have_consensus_path()` which checks descriptors. In the cold cache case, there are likely no existing descriptors, so tor will give the misleading impression that the consensus lacks exits (when it's actually more likely that it simply hasn't gotten around to downloading enough descriptors yet).
Removing the code that handles these conditions also simplifies `bootstrap_status_to_string()`.
This probably needs a tor-spec update.Tor: 0.4.0.x-finalTaylor YuTaylor Yuhttps://gitlab.torproject.org/tpo/core/tor/-/issues/27275Stop reporting appveyor on_success, because it's noisy2020-06-27T13:52:28ZteorStop reporting appveyor on_success, because it's noisyAppveyor currently reports:
```
on_success:
- cmd: ... success
on_failure:
- cmd: ... failure
```
which is really noisy.
Travis currently reports:
```
irc:
...
on_success: change
on_failure: change
```
which seems ok.
We ...Appveyor currently reports:
```
on_success:
- cmd: ... success
on_failure:
- cmd: ... failure
```
which is really noisy.
Travis currently reports:
```
irc:
...
on_success: change
on_failure: change
```
which seems ok.
We should make Appveyor notifications more like Travis.Tor: 0.3.5.x-finalhttps://gitlab.torproject.org/tpo/core/tor/-/issues/5824Stop reporting relay-only statistics as a bridge2020-06-27T14:06:23ZKarsten LoesingStop reporting relay-only statistics as a bridgeSome bridges report statistics that were originally designed for relays only. We throw these statistics away in the sanitizing process, so they don't end up in the tarballs. But the cleaner approach would be to not report them at all.
...Some bridges report statistics that were originally designed for relays only. We throw these statistics away in the sanitizing process, so they don't end up in the tarballs. But the cleaner approach would be to not report them at all.
Here's how the patch could look like; I can make a proper patch with changes file later:
```
diff --git a/src/or/config.c b/src/or/config.c
index ab4f160..2179117 100644
--- a/src/or/config.c
+++ b/src/or/config.c
@@ -1680,8 +1680,9 @@ options_act(const or_options_t *old_options)
time_t now = time(NULL);
int print_notice = 0;
- /* If we aren't acting as a server, we can't collect stats anyway. */
- if (!server_mode(options)) {
+ /* If we aren't acting as a server or if we are a bridge, we can't
+ * collect relay-only stats. */
+ if (!server_mode(options) || options->BridgeRelay) {
options->CellStatistics = 0;
options->DirReqStatistics = 0;
options->EntryStatistics = 0;
```
Is this something for 0.2.3.x or 0.2.4.x? Or do we not care at all?Tor: 0.2.5.x-finalKarsten LoesingKarsten Loesinghttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/26736Stop requiring me to change the sbws version in more than one place2020-06-27T13:41:43ZpastlyStop requiring me to change the sbws version in more than one placeWhen bumping the sbws version to do a release, I have to edit the version in two places. I'd like that to be one place.
`sbws/__init__.py`, the only place I want to have to edit the version.
`tests/unit/lib/data/v3bw/20180425_131057.v3b...When bumping the sbws version to do a release, I have to edit the version in two places. I'd like that to be one place.
`sbws/__init__.py`, the only place I want to have to edit the version.
`tests/unit/lib/data/v3bw/20180425_131057.v3bw`, the place I think it would be really nice if I didn't have to edit the version.sbws: 1.0.x-finaljugajugahttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/28458Stop resolving domains locally and check same flags for the 2nd hop2020-06-27T13:41:32ZjugaStop resolving domains locally and check same flags for the 2nd hopTo check if an exit can exit to the destination, sbws is trying to resolve the domain locally, which fails in many cases.
Even if it does not fail, the IP obtained won't be the same IP to which the exit will make the HTTP request when us...To check if an exit can exit to the destination, sbws is trying to resolve the domain locally, which fails in many cases.
Even if it does not fail, the IP obtained won't be the same IP to which the exit will make the HTTP request when using a CND.
When the domain resolution is failing, sbws try to choose other exit without restricting that does not have the bad flag and without checking that can exit to an http port.
sbws will also be faster and give less errors not resolving domains locally.sbws: 1.0.x-finaljugajuga