Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-21T18:06:14Zhttps://gitlab.torproject.org/legacy/trac/-/issues/34350Stop logging all successful databse queries in GetTor2020-06-21T18:06:14ZCecylia BocovichStop logging all successful databse queries in GetTorThis is another log message that isn't helpful and fills up our logs. Here's a patch: https://gitlab.torproject.org/tpo/anti-censorship/gettor-project/gettor/-/merge_requests/12This is another log message that isn't helpful and fills up our logs. Here's a patch: https://gitlab.torproject.org/tpo/anti-censorship/gettor-project/gettor/-/merge_requests/12Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34286gettor appears to be in an email loop war with a .sk address2020-06-21T18:06:13ZRoger Dingledinegettor appears to be in an email loop war with a .sk addressI happened to be looking at eugeni's mail.log for other debugging, and saw that approximately 25% of the lines in mail.log contain the string gettor.
(Yesterday, eugeni's postfix had 460k lines in it, and 101k of them said "gettor" in t...I happened to be looking at eugeni's mail.log for other debugging, and saw that approximately 25% of the lines in mail.log contain the string gettor.
(Yesterday, eugeni's postfix had 460k lines in it, and 101k of them said "gettor" in them. Today in the first hour or so, it's 7k out of 25k.)
Does gettor get into fights with external addresses, where it replies to the bounce, gets another bounce and replies to that, etc?
There are probably smart guidelines for avoiding mail loop wars, like not answering names that start with mailer-domain, checking for the presence of an X-Something-Something header, or rate limiting responses to a given address.
And this is a great case where unifying how bridgedb handles its email answers, and how gettor does it, will save a lot of headache.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34198Include full broker messaging spec in /doc2020-06-13T18:22:07ZCecylia BocovichInclude full broker messaging spec in /docThis adds information about the broker API, with the messaging protocol and the endpoints used by clients and proxies.
This is a prerequisite for our work to implement a Snowflake proxy on Android.This adds information about the broker API, with the messaging protocol and the endpoints used by clients and proxies.
This is a prerequisite for our work to implement a Snowflake proxy on Android.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34129Use STUN to determine NAT behaviour of peers2020-06-30T16:07:44ZCecylia BocovichUse STUN to determine NAT behaviour of peersIn investigating high proxy failure rates at clients (#33666) and the logistics of running our own STUN server (#25591), I came across [RFC5780](https://tools.ietf.org/html/rfc5780), which outlines steps to identify NATs with "endpoint i...In investigating high proxy failure rates at clients (#33666) and the logistics of running our own STUN server (#25591), I came across [RFC5780](https://tools.ietf.org/html/rfc5780), which outlines steps to identify NATs with "endpoint independent mapping and filtering".
[Section 4.3](https://tools.ietf.org/html/rfc5780#section-4.3) outlines how a client can use a STUN server with an alternate IP address (returned in the first STUN binding request response) to determine how restrictive their NAT is.
This would be useful to match up clients with snowflake proxies that have compatible NATs. We still have the following questions:
- ~~are there public STUN servers that support this feature?~~
Yes there are several candidates.
- ~~does the pion/stun library we use support this feature for STUN clients?~~
Not yet but we can implement the feature.
- If we're able to implement our own STUN server behind a domain-fronted connection (#25591), how can we implement this functionality?
I see at least one open source STUN server implementation that claims to support this (written in C): https://github.com/coturn/coturnCecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34061Reduce amount of GetTor logging2020-06-21T18:06:11ZCecylia BocovichReduce amount of GetTor loggingWe're logging at a very high level (looks like at debug), and outputting frequent successes when we only really need to be logging errors.
For example: a successfully processed email in `log/email_parser.log` outputs:
```
2020-04-27 23:...We're logging at a very high level (looks like at debug), and outputting frequent successes when we only really need to be logging errors.
For example: a successfully processed email in `log/email_parser.log` outputs:
```
2020-04-27 23:18:53+0000 [-] Log opened.
2020-04-27 23:18:53+0000 [process email] New email request received.
2020-04-27 23:18:53+0000 [process email] Reading new email.
2020-04-27 23:18:53+0000 [-] Database query executed successfully.
2020-04-27 23:18:53+0000 [email parser] Building email message from string.
2020-04-27 23:18:53+0000 [email parser] Normalizing and validating FROM email address.
2020-04-27 23:18:53+0000 [email parser] Email address normalized and validated.
2020-04-27 23:18:53+0000 [email parser] Request from [hid]
2020-04-27 23:18:53+0000 [email parser] Found request for links.
2020-04-27 23:18:53+0000 [-] Database query executed successfully.
2020-04-27 23:18:53+0000 [-] Main loop terminated.
2020-04-27 23:18:53+0000 [process email] Email request processed.
```
and in `log/gettor.log`:
```
2020-04-29T14:46:51+0000 [gettor#info] Getting links for windows is.
2020-04-29T14:46:51+0000 [-] Database query executed successfully.
2020-04-29T14:46:51+0000 [gettor#info] Sending links to [hid].
2020-04-29T14:46:51+0000 [gettor#debug] Creating plain text email
2020-04-29T14:46:51+0000 [gettor#debug] Calling asynchronous sendmail.
2020-04-29T14:46:51+0000 [twisted.mail.smtp.ESMTPSenderFactory#info] Starting factory <twisted.mail.smtp.ESMTPSenderFactory object at 0x7f0bba74b780>
2020-04-29T14:46:51+0000 [gettor#info] Email sent successfully.
2020-04-29T14:46:51+0000 [twisted.mail.smtp.ESMTPSenderFactory#info] Stopping factory <twisted.mail.smtp.ESMTPSenderFactory object at 0x7f0bba74b780>
2020-04-29T14:46:51+0000 [-] Database query executed successfully.
2020-04-29T14:46:51+0000 [-] Database query executed successfully.
```
We could reduce this to one log message at most. Especially since this information *should* be captured in the stats database.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34053Update version of wrtc packate for web proxies2020-06-13T18:22:00ZCecylia BocovichUpdate version of wrtc packate for web proxiesWe're currently using version `0.0.62`. I tried an update to the current version `0.4.4` and it works. Let's update to make these a bit more secure.We're currently using version `0.0.62`. I tried an update to the current version `0.4.4` and it works. Let's update to make these a bit more secure.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34049Fix proxy calls to session description serialization utils2020-06-13T18:21:59ZCecylia BocovichFix proxy calls to session description serialization utilsWhoops, the proxy code wasn't updated after #33897Whoops, the proxy code wasn't updated after #33897Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34035Dry out GetTor's sendmail function2020-06-21T18:06:10ZCecylia BocovichDry out GetTor's sendmail functionThis patch refactors the sendmail function in GetTor to avoid code duplication.
https://gitlab.torproject.org/torproject/anti-censorship/gettor-project/gettor/-/merge_requests/6This patch refactors the sendmail function in GetTor to avoid code duplication.
https://gitlab.torproject.org/torproject/anti-censorship/gettor-project/gettor/-/merge_requests/6Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/34027GetTor not responding to emails2020-06-21T18:06:08ZCecylia BocovichGetTor not responding to emailsGetTor isn't responding to emails. I just checked the logs and found the following errors:
```
2020-04-27T18:00:33+0000 [gettor#debug] Could not send email.
2020-04-27T18:00:33+0000 [gettor#info] Error sending email: [Failure instance: ...GetTor isn't responding to emails. I just checked the logs and found the following errors:
```
2020-04-27T18:00:33+0000 [gettor#debug] Could not send email.
2020-04-27T18:00:33+0000 [gettor#info] Error sending email: [Failure instance: Traceback (failure with no frames): <class 'twisted.mail._except.SMTPDeliveryError'>: 501 No recipients accepted
czjeff5655@!63.com: 501 5.1.3 Bad recipient address syntax
>>> MAIL FROM:<gettor@torproject.org>
<<< 250 2.1.0 Ok
>>> RCPT TO:<redacted>
<<< 501 5.1.3 Bad recipient address syntax
].
2020-04-27T18:00:33+0000 [twisted.mail.smtp.ESMTPSenderFactory#info] Stopping factory <twisted.mail.smtp.ESMTPSenderFactory object at 0x7fa3ac156940>
```
Looks like this started at `2020-04-17T16:08:57+0000`. Perhaps there was a twisted update that broke it?Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33736Update obfs4proxy package in debian2020-06-13T18:33:18ZCecylia BocovichUpdate obfs4proxy package in debianRight now the obfs4proxy package is about a year out of date. This is known and the Debian ticket for it is here: [948312](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948312).
Since more recent versions use utls, this requires som...Right now the obfs4proxy package is about a year out of date. This is known and the Debian ticket for it is here: [948312](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=948312).
Since more recent versions use utls, this requires some new packages to be added. I've filed the following itp for yawning's fork of utls: [954209](https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=954209). From what I can tell, we'll need at least two more packages as well.
I've been chipping away at this, slowly, but if someone else wants to pick it up that's great.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33693snowflake's 0.0.3.0 dummy address means rate limits are skipped means BW cont...2020-06-16T01:11:59ZRoger Dingledinesnowflake's 0.0.3.0 dummy address means rate limits are skipped means BW controller events show no bandwidth usedSnowflake's default bridge pretends to be on 0.0.3.0. It's a dummy address since snowflake-client knows how to connect to the right bridge and ignores the address that Tor tells it.
But my Tor client still uses that bridge address to ma...Snowflake's default bridge pretends to be on 0.0.3.0. It's a dummy address since snowflake-client knows how to connect to the right bridge and ignores the address that Tor tells it.
But my Tor client still uses that bridge address to make decisions. For example, connection_is_rate_limited() decides "no, it isn't rate limited", because tor_addr_is_internal() says 0.0.3.0 is essentially part of localhost. And that choice has a cascading effect where when I attach my nyx to Tor Browser to graph bandwidth use (`nyx -i 9151`), the BW events all say "0 0" because my Tor hasn't sent or received any non-internal bytes.
The quick fix is to keep using a dummy address, but to pick one that isn't an internal address. I confirmed that if I change snowflake's dummy address to 11.0.3.0, then connection_is_rate_limited() decides it's external, my BW events work again, and nyx gives me graphs. That is, Tor is smart enough to know that even though the connection is from the Tor client to the localhost snowflake client, the connection is "really" to the (non-localhost) destination bridge address.
I confess that I don't know which "apparently routable but don't worry we won't actually connect to it, probably" address is the best choice here. :/
The longer term answer is to have some other way to signal that it's a dummy address, or to change the PT interface so we don't need the fake address. But I don't think we need to wait for the longer term answer here.
The reason I noticed this issue is because I am pondering lobbying for the Tor Browser folks to give me a tiny bandwidth graph (or activity spinner) somewhere in the browser, because I got a super slow snowflake, but I was still getting 5-10KBytes/s, and my page did load after like 90 seconds, but if I hadn't been staring at the
```
2020/03/23 09:33:05 Traffic Bytes (in|out): 9018 | 10981 -- (27 OnMessages, 24 Sends)
```
lines I would have assumed that it was wedged.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33666Investigate Snowflake proxy failures2020-06-13T18:21:49ZCecylia BocovichInvestigate Snowflake proxy failuresSometimes a client will get a useless proxy from the broker. At times this happens occasionally, and at times more often. It could be a NAT problem.Sometimes a client will get a useless proxy from the broker. At times this happens occasionally, and at times more often. It could be a NAT problem.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33637Update license for Snowflake2020-06-13T18:21:46ZCecylia BocovichUpdate license for SnowflakeWorking on packaging Snowflake for debian, and perhaps our license needs an update? We should make sure we have all our ducks in a row here because Debian will care.Working on packaging Snowflake for debian, and perhaps our license needs an update? We should make sure we have all our ducks in a row here because Debian will care.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33636Remove go-webrtc dependency from snowflake2020-06-13T18:21:46ZCecylia BocovichRemove go-webrtc dependency from snowflakeWe still depend on go-webrtc because of the testing code in server-webrtc. Let's migrate this to pion or get rid of this testing code. go-webrtc is now officially unmaintained, and this will make it easier for us to package Snowflake for...We still depend on go-webrtc because of the testing code in server-webrtc. Let's migrate this to pion or get rid of this testing code. go-webrtc is now officially unmaintained, and this will make it easier for us to package Snowflake for #19409Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33543GetTor fails to respond if you don't include a proper command2020-06-21T18:06:07ZCecylia BocovichGetTor fails to respond if you don't include a proper commandRight now GetTor will only respond to emails in the following circumstances:
- With a help message if the email (or subject) contains the keyword "help"
- With links if the email (or subject) contains the keyword "links" or a valid pla...Right now GetTor will only respond to emails in the following circumstances:
- With a help message if the email (or subject) contains the keyword "help"
- With links if the email (or subject) contains the keyword "links" or a valid platform
We should fix this to **always** send an email, defaulting to a help message if a platform or links aren't provided.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33532Snowflake website does not display all available languages2020-06-13T18:21:44ZCecylia BocovichSnowflake website does not display all available languagesI just updated the snowflake website with new translations and noticed that some locales are not displayed (such as ru) even though the translations for this locale are available.
I think this is due to how `index.js` pulls languages fr...I just updated the snowflake website with new translations and noticed that some locales are not displayed (such as ru) even though the translations for this locale are available.
I think this is due to how `index.js` pulls languages from `availableLangs`. Should be a quick fix.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33364Could not connect to the bridge.2020-06-30T16:04:35ZcypherpunksCould not connect to the bridge.Console error message:
Firefox can’t establish a connection to the server at wss://snowflake.freehaven.net/.
Relevant code at: snowflake.js:867:9
A ping from the command prompt to this subdomain succeeds. It's just Firefox that can't ...Console error message:
Firefox can’t establish a connection to the server at wss://snowflake.freehaven.net/.
Relevant code at: snowflake.js:867:9
A ping from the command prompt to this subdomain succeeds. It's just Firefox that can't connect.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33343Update requirements.txt in GetTor2020-06-21T18:06:06ZCecylia BocovichUpdate requirements.txt in GetTorOur requirements.txt file is out of date which makes it difficult for devs to test things locally.
It's also worth looking at the packages in the README, since those seem to be out of date as well.Our requirements.txt file is out of date which makes it difficult for devs to test things locally.
It's also worth looking at the packages in the README, since those seem to be out of date as well.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33330Use Go modules for Snowflake2020-06-16T01:11:46ZCecylia BocovichUse Go modules for SnowflakeSnowflake CI is currently failing because we use the latest version of all libraries, the master branch of pion/dtls in particular is not compatable with its usage by other libraries.
We should add a go.mod and go.sum to snowflakeSnowflake CI is currently failing because we use the latest version of all libraries, the master branch of pion/dtls in particular is not compatable with its usage by other libraries.
We should add a go.mod and go.sum to snowflakeCecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/legacy/trac/-/issues/33306Consider mapping snowflake locales as we map the tor button and others2020-06-13T17:33:43ZemmapeelConsider mapping snowflake locales as we map the tor button and othersThe Tor Browser is based on Firefox ESR, and so it must use the locale mapping of firefox. But transifex mapping differs a bit. So we map this changes from transifex:
bn = bn-BD
en_GB = en-GB
en = en-US
es_AR = es-AR
es_CL = es-CL
es_CO...The Tor Browser is based on Firefox ESR, and so it must use the locale mapping of firefox. But transifex mapping differs a bit. So we map this changes from transifex:
bn = bn-BD
en_GB = en-GB
en = en-US
es_AR = es-AR
es_CL = es-CL
es_CO = es-CO
es = es-ES
es_MX = es-MX
fy = fy-NL
ga = ga-IE
gu = gu-IN
hi = hi-IN
hr_HR = hr-HR
hy = hy-AM
ms_MY = ms
nb = nb-NO
nn = nn-NO
nl_BE = nl-BE
pa = pa-IN
pt_BR = pt-BR
pt_PT = pt-PT
si_LK = si
sv = sv-SE
zh_CN = zh-CN
zh_HK = zh-HK
zh_TW = zh-TW
maybe Snowflake should be mapped with the same locales?Cecylia BocovichCecylia Bocovich