Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T18:04:39Zhttps://gitlab.torproject.org/legacy/trac/-/issues/34231Document and maybe improve how we're mapping TGen transfers to Tor streams/ci...2020-06-13T18:04:39ZKarsten LoesingDocument and maybe improve how we're mapping TGen transfers to Tor streams/circuitsOnionPerf uses TGen to make transfers using a local Tor client. OnionPerf also uses Stem to connect to the Tor client's control port and register for control events.
This ticket is about documenting how we can map TGen transfers to Tor ...OnionPerf uses TGen to make transfers using a local Tor client. OnionPerf also uses Stem to connect to the Tor client's control port and register for control events.
This ticket is about documenting how we can map TGen transfers to Tor streams and circuits. OnionPerf did this to produce the .tpf output format (which we just killed in #34141). But we'll also need this functionality to implement #34218 or #33260.
Here's what we're doing in [metrics-lib](https://gitweb.torproject.org/metrics-lib.git/tree/src/main/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverter.java#n121) right now to map transfers and streams:
- Index Tor circuits by their circuit ID.
- Index Tor streams by their source port; if there are two or more streams with the same source port, remember them all.
- Go through TGen transfers one by one. For each, extract the local source port.
- Go through Tor streams with the same source port and check if transfer end and stream end happened within 150 seconds.
- If there's a match, look up the corresponding circuit by circuit ID.
Note that OnionPerf took a simpler approach for producing .tpf files by remembering just one stream by source port and not applying that 150 seconds heuristic. The result was that some mappings were wrong. The approach taken by metrics-lib leads to a few missing mappings (probably as many as OnionPerf had), and apparently no wrong mappings.
Is there a way to have an exact mapping that doesn't require a heuristic? And is there a way to do it without having to wait for transfer and stream to end?https://gitlab.torproject.org/legacy/trac/-/issues/34230Update Windows toolchain for ESR 782020-06-16T01:26:28ZGeorg KoppenUpdate Windows toolchain for ESR 78We need to go over our Windows toolchain and update it wherever needed for ESR 78.We need to go over our Windows toolchain and update it wherever needed for ESR 78.https://gitlab.torproject.org/legacy/trac/-/issues/34229Update macOS toolchain for Firefox 78 ESR2020-06-16T01:26:28ZGeorg KoppenUpdate macOS toolchain for Firefox 78 ESRWe need to go over our macOS toolchain and update it wherever we think it's needed for Firefox 78 ESR.We need to go over our macOS toolchain and update it wherever we think it's needed for Firefox 78 ESR.https://gitlab.torproject.org/legacy/trac/-/issues/34228Update Linux toolchain for ESR 782020-06-16T01:26:27ZGeorg KoppenUpdate Linux toolchain for ESR 78We need to go over out Linux toolchain and update it wherever we think we need it for Firefox 78 ESR.We need to go over out Linux toolchain and update it wherever we think we need it for Firefox 78 ESR.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/legacy/trac/-/issues/34227Update desktop toolchains for ESR 782020-06-16T01:26:26ZGeorg KoppenUpdate desktop toolchains for ESR 78This is the parent ticket for all desktop toolchain related workThis is the parent ticket for all desktop toolchain related workhttps://gitlab.torproject.org/legacy/trac/-/issues/34226Investigate unused onion service error page2020-06-16T01:13:05ZMatthew FinkelInvestigate unused onion service error pageSince #19251 and #30382, Tor Browser now provides user-friendly error messages for onion service connections. But it seems like Tor Browser is not showing the correct error message for recently-online-but-offline-services. Tor should ret...Since #19251 and #30382, Tor Browser now provides user-friendly error messages for onion service connections. But it seems like Tor Browser is not showing the correct error message for recently-online-but-offline-services. Tor should return 0xF2 or 0xF3 for this situation (mostlikely 0xF3), however I'm getting the normal Firefox neterror page:
```
Hmm. We’re having trouble finding that site.
We can’t connect to the server at
```
I haven't looked at the SOCKS connection yet, so I'm not sure what is wrong.https://gitlab.torproject.org/legacy/trac/-/issues/34225Add shortName in nsIURI2020-06-16T01:13:04ZMatthew FinkelAdd shortName in nsIURIIn #28805, we integrated the https-everywhere aliasing system for (securedrop) onion addresses. I think we can simplify the patch (and future patches) by including the alias/name within the current browser instance uri (`browser.currentU...In #28805, we integrated the https-everywhere aliasing system for (securedrop) onion addresses. I think we can simplify the patch (and future patches) by including the alias/name within the current browser instance uri (`browser.currentURI`). We can achieve this by adding a `shortName` attribute (or something similar) in `netwerk/base/nsIURI.idl`. For simplicity, `host` can remain as the onion address, and we can add an additional attribute for the human-meaningful name.
(Naming is hard, but luckily this is an implementation detail so it doesn't really matter which attribute label we choose)
This will simplify patches where, in the current situation, we must pass around both the current `uri` and `onionAliasURI`.https://gitlab.torproject.org/legacy/trac/-/issues/34224Update analysis results file version to 2.02020-06-13T18:04:38ZKarsten LoesingUpdate analysis results file version to 2.0We added a new field to the analysis results file format in #26673, but we did not yet increment the format version number. The current version field is a floating point number, which doesn't work well for versioning. We should use a ver...We added a new field to the analysis results file format in #26673, but we did not yet increment the format version number. The current version field is a floating point number, which doesn't work well for versioning. We should use a version string here. Given that this change alone is going to be backward-incompatible, we'll have to call the new version 2.0. I'm preparing a patch.https://gitlab.torproject.org/legacy/trac/-/issues/34223Please create a 'translation-admin' mailing list2020-06-13T17:02:14ZMatthew FinkelPlease create a 'translation-admin' mailing listPlease create a private mailing list for 'translation-admin' without archives. I'll be the administrator (for now).
Thanks.Please create a private mailing list for 'translation-admin' without archives. I'll be the administrator (for now).
Thanks.Jens KubiezielJens Kubiezielhttps://gitlab.torproject.org/legacy/trac/-/issues/34222Create pluggable-transports/snowflake-mobile repository on git.tp.o2020-06-13T17:02:14ZCecylia BocovichCreate pluggable-transports/snowflake-mobile repository on git.tp.oWe have a new project to make a snowflake proxy for android. Keeping in line with our recent decision to split the snowflake repository, i think it makes the most sense to make a new one for this project. It will have a separate code bas...We have a new project to make a snowflake proxy for android. Keeping in line with our recent decision to split the snowflake repository, i think it makes the most sense to make a new one for this project. It will have a separate code base from the other two snowflake repositories and so a separate release cycle and process.
For now, we should give push access to at least `cohosh` and `phw`. It would be good to give `HashikD` also direct push access in the future but that may require more accounts elsewhere. For now we'll need to review the code anyway so either phw or cohosh can merge it.https://gitlab.torproject.org/legacy/trac/-/issues/34221Need permanent URL for walking onions spec2020-06-13T17:02:14ZNick MathewsonNeed permanent URL for walking onions specHi! I need a permanent URL that goes to whatever the best version of the walking onions spec is. For now could I please have http://spec.torproject.org/walking-onions redirect to https://github.com/nmathewson/walking-onions-wip/tree/ma...Hi! I need a permanent URL that goes to whatever the best version of the walking onions spec is. For now could I please have http://spec.torproject.org/walking-onions redirect to https://github.com/nmathewson/walking-onions-wip/tree/master/specs ? I'll want to change destination URL once the proposal is finished.https://gitlab.torproject.org/legacy/trac/-/issues/34220Return to stem master once stem issue 63 is resolved.2020-06-13T15:53:34ZNick MathewsonReturn to stem master once stem issue 63 is resolved.When stem fixes https://github.com/torproject/stem/issues/63 , we should revert the travis.yml change of #34204.When stem fixes https://github.com/torproject/stem/issues/63 , we should revert the travis.yml change of #34204.Tor: 0.4.4.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/34219Enable ZSTD support properly for Android2020-06-16T01:13:04ZGeorg KoppenEnable ZSTD support properly for AndroidIn #28766 we tried to enable ZSTD support for Android but just specifying `--enable-zstd` when compiling `tor` and extracting the library is not enough:
```
WARNING: Unable to find libzstd, install pkg-config, and check the PKG_CONFIG_PA...In #28766 we tried to enable ZSTD support for Android but just specifying `--enable-zstd` when compiling `tor` and extracting the library is not enough:
```
WARNING: Unable to find libzstd, install pkg-config, and check the PKG_CONFIG_PATH environment variable, or set ZSTD_CFLAGS and ZSTD_LIBS
```Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/legacy/trac/-/issues/34218Split PROXY errors into whatever reasons are given in TorCtl logs2020-06-13T18:04:37ZKarsten LoesingSplit PROXY errors into whatever reasons are given in TorCtl logsIf we combine TGen transfers with Tor streams and circuits we can learn a lot more about why our proxy (Tor) failed. Including that information in visualizations might be very useful.
When we match TGen transfers with TorCtl events, we'...If we combine TGen transfers with Tor streams and circuits we can learn a lot more about why our proxy (Tor) failed. Including that information in visualizations might be very useful.
When we match TGen transfers with TorCtl events, we'll have to look [how metrics-lib does this](https://gitweb.torproject.org/metrics-lib.git/tree/src/main/java/org/torproject/descriptor/onionperf/OnionPerfAnalysisConverter.java).
Maybe we can use some ideas from #30612 where we discussed putting an error code graph on Tor Metrics.https://gitlab.torproject.org/legacy/trac/-/issues/34217Include partial downloads in existing TTLB graphs2020-06-13T18:04:37ZKarsten LoesingInclude partial downloads in existing TTLB graphsWith #26673 being resolved we should include partial downloads in existing TTLB graphs. Maybe there's a way to put less emphasis on the different file sizes and combine the various partial downloads in a single TTLB graph.With #26673 being resolved we should include partial downloads in existing TTLB graphs. Maybe there's a way to put less emphasis on the different file sizes and combine the various partial downloads in a single TTLB graph.https://gitlab.torproject.org/legacy/trac/-/issues/34216Split visualizations into public server vs. v2 onion server vs. v3 onion serv...2020-06-13T18:04:37ZKarsten LoesingSplit visualizations into public server vs. v2 onion server vs. v3 onion server measurementsRight now, OnionPerf visualizations contain all measurements found in a data set. This can be very confusing, because this includes measurements performed via exits, via v2 onion service, and via v3 onion service. We should split visuali...Right now, OnionPerf visualizations contain all measurements found in a data set. This can be very confusing, because this includes measurements performed via exits, via v2 onion service, and via v3 onion service. We should split visualizations, or we should provide a filter and only graph one subset of measurements at a time.Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/34215Harmonize TTFB/TTLB definitions with Tor Metrics plots2020-06-13T18:04:36ZKarsten LoesingHarmonize TTFB/TTLB definitions with Tor Metrics plotsRight now, OnionPerf plots 'command' to 'first_byte', whereas Tor Metrics plots 'start' to 'first_byte' (as found in #33258). We should harmonize these definitions, which likely means using what we have been using for Tor Metrics plots f...Right now, OnionPerf plots 'command' to 'first_byte', whereas Tor Metrics plots 'start' to 'first_byte' (as found in #33258). We should harmonize these definitions, which likely means using what we have been using for Tor Metrics plots for the past decade.Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/34214Remove existing Tor control log visualizations2020-06-13T18:04:36ZKarsten LoesingRemove existing Tor control log visualizationsAs found in #33258, the existing Tor control log visualizations are not really useful to us. Let's consider removing them.
As part of this change, we should rename the TGen visualizations .pdf and .csv to something starting with `onionp...As found in #33258, the existing Tor control log visualizations are not really useful to us. Let's consider removing them.
As part of this change, we should rename the TGen visualizations .pdf and .csv to something starting with `onionperf.*`, as these files will contain parts from TGen and TorCtl in the future (error codes from TorCtl logs, measurements filtered by relay fingerprints).Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/34213Replace TorCtlParser with OnionTrace's control log parser2020-06-13T18:04:35ZKarsten LoesingReplace TorCtlParser with OnionTrace's control log parserSimilar to our plan to use TGen's analysis.py instead of OnionPerf's in #33974, the idea would be to avoid keeping our own copy of a Tor control even log parser in OnionPerf.
This work should start with an analysis how much work this is...Similar to our plan to use TGen's analysis.py instead of OnionPerf's in #33974, the idea would be to avoid keeping our own copy of a Tor control even log parser in OnionPerf.
This work should start with an analysis how much work this is going to save in the long run, and whether that justifies adding another dependency.
Feel free to adjust estimated points before starting to work on this if 2.0 doesn't make any sense.https://gitlab.torproject.org/legacy/trac/-/issues/34212Set up a domain-fronted end point for wolpertinger2020-06-13T18:36:33ZPhilipp Winterphw@torproject.orgSet up a domain-fronted end point for wolpertingerCensorship measurement platforms like OONI can request bridges to test from wolpertinger's REST API. Some platforms may be unable to talk to wolpertinger because it runs on bridges.torproject.org, which may be blocked. (This isn't an iss...Censorship measurement platforms like OONI can request bridges to test from wolpertinger's REST API. Some platforms may be unable to talk to wolpertinger because it runs on bridges.torproject.org, which may be blocked. (This isn't an issue for OONI because it intends to proxy wolpertinger requests over its backend.)
We should set up a domain-fronted endpoint to fix this issue: In addition to a direct connection to bridges.torproject.org/wolpertinger, censorship measurement platforms should be able to use a domain front at ajax.aspnetcdn.com.
After reading #27469 and #16650, I believe that we need to configure another azure reflector, e.g., wolpertinger.azureedge.net, which is hooked up to https://bridges.torproject.org/wolpertinger/.Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.org