Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-21T18:05:48Zhttps://gitlab.torproject.org/legacy/trac/-/issues/28234Update GetTor documentation2020-06-21T18:05:48ZtraumschuleUpdate GetTor documentationThe documentation should be updated at several places.The documentation should be updated at several places.traumschuletraumschulehttps://gitlab.torproject.org/legacy/trac/-/issues/28152Gettor code refactor with Python Twisted2020-06-21T18:05:45ZIsrael LeivaGettor code refactor with Python TwistedCode refactor
Gettor needs some love. It should be more robust to make it: easier to maintain (by me or somebody else), to know when it is working or not, and to allow more developers to contribute to it.
For the above, I propose to re...Code refactor
Gettor needs some love. It should be more robust to make it: easier to maintain (by me or somebody else), to know when it is working or not, and to allow more developers to contribute to it.
For the above, I propose to refactor the current code and turn it into a twisted daemon [1, 2]. This would preserve the main logic of the current system and add all the benefits of having a daemonized application. This service approach considers two main parts:
1. Distribution channels. Whenever gettor receives a request or sends a reply it uses a channel (e.g. e-mail). Each channel could be handled by one or more services. These services would be constantly fetching and updating information in a SQLite database to know how to proceed.
In the case of e-mail, there should be a script that receives messages forwarded by the MTA, process them, and add a request with a given status to the SQLite database. On the other hand, a service running on background will be fetching ready-to-be-sent requests from the database and send e-mails with the requested information.
For a twitter bot, a single service that receives DMs, process them and send replies would be enough.
2. Tor Browser sync. A service constantly checking new Tor Browser releases, downloading the new packages and updating the SQLite database with the new links.
The logging system provided by twistd is easy to use and works very well. This will solve one of the problems with the current code and the use of logging, also providing useful information for debugging and statistics. Log rotation is automatic.
I have developed a similar service using twistd. Adapting it to gettor would be fairly easy and it would take me a few weeks of spare time.
Twisted is not installed on getulum, so I will collect all the needed packages and ask for them to be installed.
1: https://twistedmatrix.com/documents/current/core/howto/application.html.
2: https://twistedmatrix.com/documents/current/core/howto/basics.html#twistdHiroHirohttps://gitlab.torproject.org/legacy/trac/-/issues/28091Port GetTor to python32020-06-21T18:05:44ZtraumschulePort GetTor to python3It's good to be ahead of time refactoring GetTor to python3.
https://docs.python.org/3/howto/pyporting.html
https://docs.python.org/2/library/2to3.html#to3-referenceIt's good to be ahead of time refactoring GetTor to python3.
https://docs.python.org/3/howto/pyporting.html
https://docs.python.org/2/library/2to3.html#to3-referencetraumschuletraumschulehttps://gitlab.torproject.org/legacy/trac/-/issues/14744Automate upload of latest Tor Browser to cloud services2020-06-21T18:05:15ZIsrael LeivaAutomate upload of latest Tor Browser to cloud servicesCurrently, to have the latest Tor Browser version delivered is necessary to manually upload the files every time a new version of Tor Browser is released. This could easily be automated thanks to [RecommendedTBBVersions](https://www.torp...Currently, to have the latest Tor Browser version delivered is necessary to manually upload the files every time a new version of Tor Browser is released. This could easily be automated thanks to [RecommendedTBBVersions](https://www.torproject.org/projects/torbrowser/RecommendedTBBVersions). This will help to avoid the deliver of old Tor Browser versions (see #12502). A preliminary script for this can be found [here](https://github.com/ilv/gettor/blob/develop/upload/fetch_latest_torbrowser.py).https://gitlab.torproject.org/legacy/trac/-/issues/1593Implement test (-t switch) functionality2020-06-21T18:04:40ZAndrew LewmanImplement test (-t switch) functionalityImplement test (-t switch) functionalityImplement test (-t switch) functionalityChristian FrommeChristian Frommehttps://gitlab.torproject.org/legacy/trac/-/issues/29024Add pluggable-transport support to Chutney2020-06-13T18:36:14ZNick MathewsonAdd pluggable-transport support to ChutneyWe need to make PTs in general, and Snowflake in particular, more reliable and well-tested. On way to do that is with realistic integration tests, using Chutney.We need to make PTs in general, and Snowflake in particular, more reliable and well-tested. On way to do that is with realistic integration tests, using Chutney.Tor: 0.4.2.x-finalNick MathewsonNick Mathewsonhttps://gitlab.torproject.org/legacy/trac/-/issues/30472Implement a mechanism for PT reachability testing2020-06-13T18:35:41ZPhilipp Winterphw@torproject.orgImplement a mechanism for PT reachability testingNon-vanilla bridges currently have no way to automatically test their reachability. Vanilla bridges [self-test the reachability of their ORPort](https://gitweb.torproject.org/torspec.git/tree/path-spec.txt#n193) by creating a circuit tha...Non-vanilla bridges currently have no way to automatically test their reachability. Vanilla bridges [self-test the reachability of their ORPort](https://gitweb.torproject.org/torspec.git/tree/path-spec.txt#n193) by creating a circuit that includes themselves, but we cannot do this for, say, obfs4. In practice, this is problematic because obfs4 operators won't know if their bridge is unreachable; for example due to NAT. In fact, BridgeDB is distributing obfs4 bridges that aren't actually reachable.
We need to build a mechanism that allows non-vanilla bridges to test their reachability. Ideally, something would create a circuit over the bridge while speaking its respective transport protocol but even a simple TCP or UDP-based reachability test would already go a long way.
Looking at the discussion [over in #30331](https://trac.torproject.org/projects/tor/ticket/30331#comment:2), tor seems to be the right component to trigger the reachability test. In its log files, it can then yell at the operator if the test failed. The question is: how should we design the mechanism that implements the reachability test?
One solution would be a simple HTTP API that takes as input an address, port, a transport type, and optional parameters, and then tells you if the given bridge is reachable, e.g.: the URL https://pt-reachable.torproject.org/obfs4/1.2.3.4/9002 may respond with something along the lines of `obfs4_reachable: true`. Ideally, if the reachability test fails, we should provide details, to help the operator figure out what went wrong.Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/30331obfs4_bridgeline.txt file should contain complete bridge line2020-06-13T18:35:39ZCecylia Bocovichobfs4_bridgeline.txt file should contain complete bridge lineWhen setting up an obfs4 bridge, the user has to perform extra steps to fill in the missing values to construct the full bridge line from `/var/lib/tor/pt_state/obfs4_bridgeline.txt`.
Specifically in:
`Bridge obfs4 <IP ADDRESS>:<PORT> <...When setting up an obfs4 bridge, the user has to perform extra steps to fill in the missing values to construct the full bridge line from `/var/lib/tor/pt_state/obfs4_bridgeline.txt`.
Specifically in:
`Bridge obfs4 <IP ADDRESS>:<PORT> <FINGERPRINT> cert=<CERTIFICATE> iat-mode=0`
only `cert` is populated automaticallyhttps://gitlab.torproject.org/legacy/trac/-/issues/29284Deploy Marionette as a PT2020-06-13T18:35:34ZCecylia BocovichDeploy Marionette as a PTActually get Marionette integrated with Tor. This depends on first assessing it (#29272)Actually get Marionette integrated with Tor. This depends on first assessing it (#29272)George KadianakisGeorge Kadianakishttps://gitlab.torproject.org/legacy/trac/-/issues/29278Assess HTTP proxy2020-06-13T18:35:33ZCecylia BocovichAssess HTTP proxyLook at the status of HTTP proxy and see what it will take to integrate it with Tor.Look at the status of HTTP proxy and see what it will take to integrate it with Tor.Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/28181spec: Add to pt-spec.txt control messages going back to main process (tor)2020-06-13T18:35:29ZDavid Gouletdgoulet@torproject.orgspec: Add to pt-spec.txt control messages going back to main process (tor)As part of #28180, we want to have a PT to report back events, mainly connection to the bridge events for now, to the parent process which is tor in our case.
This ticket is for adding those to pt-spec.txt.
We'll concentrate on connect...As part of #28180, we want to have a PT to report back events, mainly connection to the bridge events for now, to the parent process which is tor in our case.
This ticket is for adding those to pt-spec.txt.
We'll concentrate on connection messages reporting the status of a connection to a Bridge at first to narrow down a bit this work.Tor: 0.4.0.x-finalDavid Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/28940Add support for LOG to goptlib2020-06-13T18:35:29ZDavid Fifielddcf@torproject.orgAdd support for LOG to goptlibsee:
* #28179 (code changes)
* #28181 (pt-spec changes) _[doesn't seem to be committed yet?]_
ahf made a branch here:
https://github.com/ahf/goptlib/commits/features/loggingsee:
* #28179 (code changes)
* #28181 (pt-spec changes) _[doesn't seem to be committed yet?]_
ahf made a branch here:
https://github.com/ahf/goptlib/commits/features/loggingDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/28936Use Travis CI for goptlib.git repositories on Github2020-06-13T18:35:28ZAlexander Færøyahf@torproject.orgUse Travis CI for goptlib.git repositories on GithubMembers on the network team have been happy to use the Travis CI for `tor.git` in the past year or so.
Let's have the same for `goptlib.git` if some people are going to do development there and have their repositories located on Github.Members on the network team have been happy to use the Travis CI for `tor.git` in the past year or so.
Let's have the same for `goptlib.git` if some people are going to do development there and have their repositories located on Github.Tor: unspecifiedDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/25502Report intermediate PT bootstrapping status2020-06-13T18:35:28ZAlexander Færøyahf@torproject.orgReport intermediate PT bootstrapping statusParent ticket for all tickets about reporting intermediate PT bootstrap progress.Parent ticket for all tickets about reporting intermediate PT bootstrap progress.Tor: 0.4.0.x-finalAlexander Færøyahf@torproject.orgAlexander Færøyahf@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/29483Use systemd init script for BridgeDB2020-06-13T18:29:27ZDavid Gouletdgoulet@torproject.orgUse systemd init script for BridgeDBThe bridgedb process is executed in a cron at bootup. So if it crashes, we do not know about it because lack of monitoring but also it won't be restarted.
Lets move this out of the cron and into a systemd init script. The machine is Deb...The bridgedb process is executed in a cron at bootup. So if it crashes, we do not know about it because lack of monitoring but also it won't be restarted.
Lets move this out of the cron and into a systemd init script. The machine is Debian 9.7 so systemd is stable there and what should be used.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/29481Cleanup bridgedb.conf and bridgedb.crontab2020-06-13T18:29:27ZDavid Gouletdgoulet@torproject.orgCleanup bridgedb.conf and bridgedb.crontabThe production `bridgedb.conf` needs to be cleaned up due to several outdated config in there.The production `bridgedb.conf` needs to be cleaned up due to several outdated config in there.Matthew FinkelMatthew Finkelhttps://gitlab.torproject.org/legacy/trac/-/issues/29273Document BridgeDB infrastructure2020-06-13T18:29:23ZAlexander Færøyahf@torproject.orgDocument BridgeDB infrastructureWe should document how the current BridgeDB that is running is configured and how to configure a new instance in case there is a problem with the one we have today.We should document how the current BridgeDB that is running is configured and how to configure a new instance in case there is a problem with the one we have today.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/29229Does anybody notice if the bridge auth goes away?2020-06-13T18:29:22ZGabagaba@torproject.orgDoes anybody notice if the bridge auth goes away?Set some kind of notification system for bridge auth.Set some kind of notification system for bridge auth.David Gouletdgoulet@torproject.orgDavid Gouletdgoulet@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/28655If a bridge supports obfs4, don't give out its other flavors2020-06-13T18:29:20ZRoger DingledineIf a bridge supports obfs4, don't give out its other flavorsThere's a FOCI 2018 paper looking at blocking of bridges inside China, and one of their conclusions is that China has moved from "block by IP:port" to "block to IP":
https://www.usenix.org/conference/foci18/presentation/dunna
If that is...There's a FOCI 2018 paper looking at blocking of bridges inside China, and one of their conclusions is that China has moved from "block by IP:port" to "block to IP":
https://www.usenix.org/conference/foci18/presentation/dunna
If that is so, it means that when bridgedb gives out the vanilla ORPort of an obfs4 bridge, then some user will get it, try to use it from inside China, trigger the active probing, and get the whole IP address blocked -- including the obfs4 port.
The fix: when bridgedb gets a bridge that supports an active-probing resistant transport (right now that means obfs4), it needs to decide not to give out the other transports for that bridge (vanilla ORPort, obfs3, etc).
(There are two caveats for this plan. First, it means we're prioritizing obfs4 bridges for the China context, since all of these transports will still be useful for countries other than China. I'm ok with that. Second, it assumes that the FOCI paper is actually correct in its conclusions about how China has changed its blocking. I recall in the Q&A at the end of the presentation that some folks questioned the analysis, but I didn't follow it enough to form a solid opinion. But even if China isn't doing its censorship in this new way yet, now is a great time for bridgedb to become able to handle it.)Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/26154Remove apt-get update from BridgeDB's .travis.yml to avoid SHA1 signature error2020-06-13T18:29:13ZIsis LovecruftRemove apt-get update from BridgeDB's .travis.yml to avoid SHA1 signature errore.g. https://travis-ci.org/isislovecruft/bridgedb/jobs/381817517#L574e.g. https://travis-ci.org/isislovecruft/bridgedb/jobs/381817517#L574Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.org