Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T14:17:58Zhttps://gitlab.torproject.org/legacy/trac/-/issues/5304Obfsproxy should respect OutboundBindAddress in torrc2020-06-13T14:17:58ZTracObfsproxy should respect OutboundBindAddress in torrcRather it just binds to * (any IP).
Tested with latest git obfsproxy and Tor 0.2.3.12-alpha.
**Trac**:
**Username**: korobkovRather it just binds to * (any IP).
Tested with latest git obfsproxy and Tor 0.2.3.12-alpha.
**Trac**:
**Username**: korobkovTor: 0.4.4.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/21315publish some realtime stats from the broker?2020-06-13T18:20:25ZRoger Dingledinepublish some realtime stats from the broker?How many snowflakes are there registered right now and happy to serve censored users?
Right now there's a big difference between 0 and 1, and it's not easy to figure out which it is.
Knowing this number would help me as a snowflake vol...How many snowflakes are there registered right now and happy to serve censored users?
Right now there's a big difference between 0 and 1, and it's not easy to figure out which it is.
Knowing this number would help me as a snowflake volunteer decide whether I am needed, and whether to do advocacy at this moment to get other people to be snowflakes.
Knowing this number would help the censored users too, because it would give them a sense of the health of the snowflake population, and also it can help them debug their "it's not working, I wonder if I can narrow down some possible problems" situations.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/23225GetTor should ignore quoted keywords in email replies that quote the help mes...2020-06-21T18:05:37ZTaylor YuGetTor should ignore quoted keywords in email replies that quote the help messageIf I send an email with the subject "help" to GetTor, and then send an "blank reply that quotes the entire message, I get the links for OrBot for Android (presumably because that's the first recognized keyword in the quoted reply. It's ...If I send an email with the subject "help" to GetTor, and then send an "blank reply that quotes the entire message, I get the links for OrBot for Android (presumably because that's the first recognized keyword in the quoted reply. It's not clear to me what a "blank" reply should produce in response to a help message, maybe a more detailed help message? If there is no more detailed help available, maybe don't mention the blank message option?Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/23226GetTor help message could be more helpful2020-06-21T18:05:38ZTaylor YuGetTor help message could be more helpfulThe GetTor email help message says
```
This is how you can request a tor browser bundle link.
Send an email to: gettor@torproject.org
In the body of the email only write: <operating system> <language>.
We only support windows, osx and...The GetTor email help message says
```
This is how you can request a tor browser bundle link.
Send an email to: gettor@torproject.org
In the body of the email only write: <operating system> <language>.
We only support windows, osx and linux as operating systems.
```
Some things we can add:
- instructions on how to verify the signature
- instructions on how to get bridges?Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/27330@get_tor on twitter not responding2020-06-21T18:05:42Zstephw@get_tor on twitter not respondingnot responding to DMsnot responding to DMsHiroHirohttps://gitlab.torproject.org/legacy/trac/-/issues/28152Gettor code refactor with Python Twisted2020-06-21T18:05:45ZIsrael LeivaGettor code refactor with Python TwistedCode refactor
Gettor needs some love. It should be more robust to make it: easier to maintain (by me or somebody else), to know when it is working or not, and to allow more developers to contribute to it.
For the above, I propose to re...Code refactor
Gettor needs some love. It should be more robust to make it: easier to maintain (by me or somebody else), to know when it is working or not, and to allow more developers to contribute to it.
For the above, I propose to refactor the current code and turn it into a twisted daemon [1, 2]. This would preserve the main logic of the current system and add all the benefits of having a daemonized application. This service approach considers two main parts:
1. Distribution channels. Whenever gettor receives a request or sends a reply it uses a channel (e.g. e-mail). Each channel could be handled by one or more services. These services would be constantly fetching and updating information in a SQLite database to know how to proceed.
In the case of e-mail, there should be a script that receives messages forwarded by the MTA, process them, and add a request with a given status to the SQLite database. On the other hand, a service running on background will be fetching ready-to-be-sent requests from the database and send e-mails with the requested information.
For a twitter bot, a single service that receives DMs, process them and send replies would be enough.
2. Tor Browser sync. A service constantly checking new Tor Browser releases, downloading the new packages and updating the SQLite database with the new links.
The logging system provided by twistd is easy to use and works very well. This will solve one of the problems with the current code and the use of logging, also providing useful information for debugging and statistics. Log rotation is automatic.
I have developed a similar service using twistd. Adapting it to gettor would be fairly easy and it would take me a few weeks of spare time.
Twisted is not installed on getulum, so I will collect all the needed packages and ask for them to be installed.
1: https://twistedmatrix.com/documents/current/core/howto/application.html.
2: https://twistedmatrix.com/documents/current/core/howto/basics.html#twistdHiroHirohttps://gitlab.torproject.org/legacy/trac/-/issues/29367Port OnionPerf to Python 32020-06-13T18:03:49ZirlPort OnionPerf to Python 3Currently OnionPerf is written in Python 2, which is soon to be end-of-life.
(Copied from https://github.com/robgjansen/onionperf/issues/41)Currently OnionPerf is written in Python 2, which is soon to be end-of-life.
(Copied from https://github.com/robgjansen/onionperf/issues/41)Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/legacy/trac/-/issues/29369Fix message logging and filtering2020-06-13T18:03:50ZirlFix message logging and filteringWhen running in measurement mode, new analysis results are produced every night at midnight utc. However, in cases when OnionPerf was not running at midnight and restarts on a later day, log messages from both partial days appear in the ...When running in measurement mode, new analysis results are produced every night at midnight utc. However, in cases when OnionPerf was not running at midnight and restarts on a later day, log messages from both partial days appear in the same log file. When the analysis is run at the midnight following the day it was restarted, downloads from the day it was first stopped may appear in the analysis results (the .tpf files).
We should fix this.
allow onionperf analysis mode to filter out log messages that do not match a certain date; onionperf measurement mode should take advantage of that feature
When exporting Torperf .tpf files, and a date filter is applied, we need to ensure that we do not include downloads that started on a different day (to handle the case where it ended on the filtered date but started on the previous day)
onionperf should name the log files according to the date of the log message timestamps, and each log message should only be appended to the file corresponding to that date
(Copied from https://github.com/robgjansen/onionperf/issues/31)Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/legacy/trac/-/issues/30368Run some tests to check reachability of snowflake proxies2020-06-13T18:20:01ZCecylia BocovichRun some tests to check reachability of snowflake proxiesOur standalone proxies were recently blocked in China: #30350
We should start running some probe tests like we are for obfs4 to see whether this blocking was a one-off event and detect blocking of new proxy instances.Our standalone proxies were recently blocked in China: #30350
We should start running some probe tests like we are for obfs4 to see whether this blocking was a one-off event and detect blocking of new proxy instances.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/30731Publish post-sanitization broker logs2020-06-13T18:20:17ZDavid Fifielddcf@torproject.orgPublish post-sanitization broker logs#30693 deleted old unsanitized server logs. Before deletion, I extracted a sanitized CSV file that is enough to make [[comment:3:ticket:30693|graphs like this]].
This ticket is to discuss whether the sanitized CSV is safe to publish and...#30693 deleted old unsanitized server logs. Before deletion, I extracted a sanitized CSV file that is enough to make [[comment:3:ticket:30693|graphs like this]].
This ticket is to discuss whether the sanitized CSV is safe to publish and publish it if so. The sanitized log is currently on the snowflake-broker host under the filename:
/var/log/snowflake-broker/broker.csv.xz
The scripts used to create it are
[broker-logs.zip:ticket:30693](None/broker-logs.zip:ticket:30693)https://gitlab.torproject.org/legacy/trac/-/issues/30830Clean up snowflake broker logs2020-06-13T18:20:18ZCecylia BocovichClean up snowflake broker logsWe recently produces graphs from the unsanitized broker logs [here](https://trac.torproject.org/projects/tor/ticket/30693#comment:3). However, the script to produce these graphs was complicated due to the structure of the log messages. W...We recently produces graphs from the unsanitized broker logs [here](https://trac.torproject.org/projects/tor/ticket/30693#comment:3). However, the script to produce these graphs was complicated due to the structure of the log messages. We should evaluate log output from the broker and decide what to keep, what to change, and what to discard.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/31253Add a webext packaging target to the build script2020-06-13T18:20:45ZArlo BreaultAdd a webext packaging target to the build scriptRun `npm run pack-webext -- 0.0.x`
I imagine that'll do something like,
```
git clean -x -d -f webext/
require('webext/manifest.json').version = argv[1];
git commit -m "bump version to ${argv[1]}"
git tag argv[1]
git push --tags
npm ru...Run `npm run pack-webext -- 0.0.x`
I imagine that'll do something like,
```
git clean -x -d -f webext/
require('webext/manifest.json').version = argv[1];
git commit -m "bump version to ${argv[1]}"
git tag argv[1]
git push --tags
npm run webext
zip webext
```
See comment:6:ticket:31087 for the original suggestion.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/31310Refactor/remove proxy-pair state machine in webextension2020-06-13T18:20:48ZCecylia BocovichRefactor/remove proxy-pair state machine in webextensionThe snowflake badge and webextension code is heavily based off of flash proxy, but the proxy pair state machine, which used attributes such as "running" and "active" doesn't map as nicely to the webrtc context. It would be nice to refact...The snowflake badge and webextension code is heavily based off of flash proxy, but the proxy pair state machine, which used attributes such as "running" and "active" doesn't map as nicely to the webrtc context. It would be nice to refactor this to be simpler, more closely match our understanding of the webrtc state machine and be easier to maintain. This might mean removing these attributes entirely and using the existence of proxy pairs to indicate if there are any open or running connections (see https://trac.torproject.org/projects/tor/ticket/31100#comment:23).Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/31376Make a /metrics handle at the snowflake broker for the stats collector2020-06-13T18:20:48ZCecylia BocovichMake a /metrics handle at the snowflake broker for the stats collectorWe have metrics being written to a log file for now as specified in #21315, and the metrics team is working on setting up a collector module for it in #29461. We should add a handler at the broker at `snowflake-broker@bamsoftware.com/met...We have metrics being written to a log file for now as specified in #21315, and the metrics team is working on setting up a collector module for it in #29461. We should add a handler at the broker at `snowflake-broker@bamsoftware.com/metrics` so the collector can make GET requests for the collected metrics.
The response should be
`Content-Type: text/plain; charset=utf-8`Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/31384localize snowflake website2020-06-13T18:20:50Zemmapeellocalize snowflake websitehttps://snowflake.torproject.org/ should appear at least on our priority 12 languages.https://snowflake.torproject.org/ should appear at least on our priority 12 languages.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/31407Create a broker spec for metrics collection2020-06-13T18:20:53ZCecylia BocovichCreate a broker spec for metrics collectionWe're in the process of creating a module for the collection of snowflake metrics (#21315, #29461). We need a better place to put the spec for the metrics data output by the snowflake broker than a comment in the source code (see https:/...We're in the process of creating a module for the collection of snowflake metrics (#21315, #29461). We need a better place to put the spec for the metrics data output by the snowflake broker than a comment in the source code (see https://trac.torproject.org/projects/tor/ticket/29461#comment:5)
A spec for the broker will also be useful to expand upon later to specify how the broker interacts with other pieces of either Snowflake or the Tor ecosystem in the case that the broker assumes more of the responsibilities of BridgeDB.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/31701Reachability tests for new obfs4 bridges2020-06-13T18:33:13ZCecylia BocovichReachability tests for new obfs4 bridgesAs a follow up to #29279, we can now set up some new reachability tests on a subset of the bridges we've gotten through our bridge campaign \o/
We probably don't want to test all of the new bridges in case these tests cause a bunch of b...As a follow up to #29279, we can now set up some new reachability tests on a subset of the bridges we've gotten through our bridge campaign \o/
We probably don't want to test all of the new bridges in case these tests cause a bunch of bridges to get blocked when they otherwise wouldn't.
As mentioned in [#29279:comment:9](https://trac.torproject.org/projects/tor/ticket/29279#comment:9), we should sample bridges from our various distribution mechanisms (email, private, and HTTPS), and also from any finer grained partitions we have (email provider, subnet, etc.).Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/31843Make safelogger thread safe2020-06-13T18:21:05ZCecylia BocovichMake safelogger thread safeIt would be nice to pass the output of the safe logger to libraries so that we can log errors that occur in library functions. Right now the safelogger is not thread safe. Multiple calls to Write from different threads results in race co...It would be nice to pass the output of the safe logger to libraries so that we can log errors that occur in library functions. Right now the safelogger is not thread safe. Multiple calls to Write from different threads results in race conditions.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/32480Use Github/Gitlab releases to distribute gettor binaries2020-06-21T18:05:56ZCecylia BocovichUse Github/Gitlab releases to distribute gettor binariesRight now GetTor attempts to delete the previous git release branch and upload a new one due to [limits on repository sizes](https://help.github.com/en/github/managing-large-files/what-is-my-disk-quota#file-and-repository-size-limitation...Right now GetTor attempts to delete the previous git release branch and upload a new one due to [limits on repository sizes](https://help.github.com/en/github/managing-large-files/what-is-my-disk-quota#file-and-repository-size-limitations).
Perhaps a better way to handle this would be to set up [Github releases](https://help.github.com/en/github/administering-a-repository/about-releases). This can be done through the [REST API](https://developer.github.com/v3/repos/releases/).Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/32576Fix race condition in snowflake broker2020-06-13T18:21:17ZCecylia BocovichFix race condition in snowflake brokerThere is a race condition with the snowflake heap that has been causing the broker to crash several times a day. This race condition has existed in the broker for several years, but some recent updates as well as the host migration manag...There is a race condition with the snowflake heap that has been causing the broker to crash several times a day. This race condition has existed in the broker for several years, but some recent updates as well as the host migration managed to shake it loose.
----
This race condition is causing the snowflake broker to crash repeatedly and often since the migration. We noticed because CollecTor stopped collecting metrics since the restart on 14 November 2019.Cecylia BocovichCecylia Bocovich