Anti-censorship issueshttps://gitlab.torproject.org/groups/tpo/anti-censorship/-/issues2020-06-27T13:42:44Zhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/33631BridgeDB doesn't allow bridges to change their distribution mechanism2020-06-27T13:42:44ZPhilipp Winterphw@torproject.orgBridgeDB doesn't allow bridges to change their distribution mechanismIf a bridge configures `BridgeDistribution https` and then changes its mind and configures `BridgeDistribution any`, BridgeDB won't allow that and keeps thinking that the bridge wants to be distributed over HTTPS. The offending code is [...If a bridge configures `BridgeDistribution https` and then changes its mind and configures `BridgeDistribution any`, BridgeDB won't allow that and keeps thinking that the bridge wants to be distributed over HTTPS. The offending code is [here](https://gitweb.torproject.org/bridgedb.git/tree/bridgedb/Bridges.py?id=9a7ae57cfc85be8aa488bf0e69e8b6f35e2ce38d#n497).
This surprised me and I would consider it unexpected behaviour. Bridges should be able to change their distribution mechanism any time.Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/33388Patch for #31967. Changed pseudo-random generator to random.SystemRandom.choi...2020-06-27T13:42:45ZArmin HuremagicPatch for #31967. Changed pseudo-random generator to random.SystemRandom.choice()From fc8b3c59d23b2cc637e4db5cd8385720027f59e0 Mon Sep 17 00:00:00 2001
From: agix <columbeff@gmail.com>
Date: Thu, 20 Feb 2020 01:24:24 +0100
Subject: [PATCH] Fix for legacy/trac#31967. Changed pseudo-random generator to
random.SystemRa...From fc8b3c59d23b2cc637e4db5cd8385720027f59e0 Mon Sep 17 00:00:00 2001
From: agix <columbeff@gmail.com>
Date: Thu, 20 Feb 2020 01:24:24 +0100
Subject: [PATCH] Fix for legacy/trac#31967. Changed pseudo-random generator to
random.SystemRandom.choice()
---
bridgedb/captcha.py | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/bridgedb/captcha.py b/bridgedb/captcha.py
index b66972c..485974b 100644
--- a/bridgedb/captcha.py
+++ b/bridgedb/captcha.py
@@ -386,7 +386,7 @@ class GimpCaptcha(Captcha):
and a challenge string (used for checking the client's solution).
"""
try:
- imageFilename = random.choice(os.listdir(self.cacheDir))
+ imageFilename = random.SystemRandom().choice(os.listdir(self.cacheDir))
imagePath = os.path.join(self.cacheDir, imageFilename)
with open(imagePath) as imageFile:
self.image = imageFile.read()
--
2.17.1https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/33299Remove retired pluggable transports from BridgeDB2021-07-01T17:47:15ZPhilipp Winterphw@torproject.orgRemove retired pluggable transports from BridgeDBBridgeDB still hands out obfs3, ScrambleSuit, and FTE bridges. Tor Browser no longer supports FTE (see legacy/trac#29319), so we should remove it. I suggest also removing obfs3 and ScrambleSuit because these transports don't offer anythi...BridgeDB still hands out obfs3, ScrambleSuit, and FTE bridges. Tor Browser no longer supports FTE (see legacy/trac#29319), so we should remove it. I suggest also removing obfs3 and ScrambleSuit because these transports don't offer anything that obfs4 doesn't already provide.
As of today, BridgeDB knows about 1,316 bridges. Among these:
* 31 support FTE. 29 of these wouldn't be handed out because they also support obfs4 (see legacy/trac#28655). The remaining two bridges run FTE/obfs3 and ScrambleSuit/obfs3/FTE.
* 34 support ScrambleSuit. 32 of these also support obfs4 and only two don't. Instead, they run obfs3/ScrambleSuit and ScrambleSuit/obfs3/FTE.
* 106 support obfs3. Only seven of these don't support obfs4.
Considering the above, I think it's safe to retire FTE, ScrambleSuit, and obfs3.Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32997Captcha error on bridges.torproject.org2020-06-27T13:42:45ZTracCaptcha error on bridges.torproject.orgMultiple times I try to get bridges even with tor connection the bridges service show captcha repeatedly...
Even after about 20 times with correct complete captcha do not show the result!!!
Please review this process and even change th...Multiple times I try to get bridges even with tor connection the bridges service show captcha repeatedly...
Even after about 20 times with correct complete captcha do not show the result!!!
Please review this process and even change the captcha image generator...
That is a bad captcha which does not have different between upper and lower case of some letters.
**Trac**:
**Username**: amirhosseinhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32900Reimplement and generalise BridgeDB?2020-09-02T18:05:21ZPhilipp Winterphw@torproject.orgReimplement and generalise BridgeDB?Over at legacy/trac#30946, I spent several hours trying to port BridgeDB to Python 3 but progress has been frustratingly slow. Given the port and our in-progress grant application proposing a social bridge distributor, I started thinking...Over at legacy/trac#30946, I spent several hours trying to port BridgeDB to Python 3 but progress has been frustratingly slow. Given the port and our in-progress grant application proposing a social bridge distributor, I started thinking about a rewrite of BridgeDB. Here's how I see the (dis)advantages breaking down:
Disadvantages:
* Re-implementations are never as smooth and straightforward as anticipated. It will take a lot of time.
* We won't (easily) be able to use Stem to parse bridge descriptors. We could extend [zoossh](https://gitweb.torproject.org/user/phw/zoossh.git/) though.
Advantages:
* We could re-implement the service in golang because the anti-censorship team is comfortable in the language.
* We could generalise the concept of BridgeDB: What goes in should be an abstract type of proxy (be it bridge descriptors, snowflake-style proxy registrations, or maybe even Lantern proxies) and distributors (as we already have them in BridgeDB) determine who gets these proxies.
* We would design the service with redundancy and "containerisation" in mind.
* It would make it easier to add new features, especially significant ones, like a new distributor.
* A re-implementation may be a return on investment and save us time in the long run.
Note that we don't need to abandon BridgeDB and then redirect our focus to its re-implementation. I would instead like to spend some hours experimenting with a new design in parallel to maintaining BridgeDB. We also don't need to aim for a feature-complete replacement of BridgeDB. For example, we don't need to PGP-sign emails. If all of the above proves fruitful, we can gradually transition to the new design.
Thoughts? Any other features or architectural changes we should make in a re-implementation?Sponsor 30 - Objective 2.3Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32740Implement a feedback loop between BridgeDB and OONI2020-10-29T17:58:29ZPhilipp Winterphw@torproject.orgImplement a feedback loop between BridgeDB and OONIBridgeDB is unaware of where its bridges are blocked. This means that if bridge 1.2.3.4 is blocked in Iran, BridgeDB will still hand it out to users from Iran, who will find themselves unable to use the bridge.
To fix this problem, we s...BridgeDB is unaware of where its bridges are blocked. This means that if bridge 1.2.3.4 is blocked in Iran, BridgeDB will still hand it out to users from Iran, who will find themselves unable to use the bridge.
To fix this problem, we should implement a feedback loop between BridgeDB and OONI. BridgeDB should hand the bridge 1.2.3.4 to OONI and have it tested in as many countries as possible. The result should then make it back to BridgeDB, so BridgeDB knows that 1.2.3.4 shouldn't be handed out to people in Iran.
OONI already has GitHub issues for this project:
* https://github.com/ooni/orchestra/issues/51
* https://github.com/ooni/orchestra/issues/68
There are several open questions:
* How does BridgeDB share a bridge with OONI?
* How should BridgeDB decide what bridges it wants tested, and how often?
* How should OONI's test results feed back into BridgeDB?
* How can we implement this feedback loop without making bridges easier to enumerate?
Child issues:
* tpo/anti-censorship/bridgedb#34116
* tpo/anti-censorship/wolpertinger#34212
* tpo/tpa/team#34089
* tpo/anti-censorship/bridgedb#34260
* tpo/anti-censorship/wolpertinger#34259
* tpo/anti-censorship/wolpertinger#34195Sponsor 30 - Objective 2.3Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32662Rewrite BridgeDB with Django 3 and Python 32020-06-27T13:42:45ZTracRewrite BridgeDB with Django 3 and Python 3BridgeDB, to my knowledge, was developed more than 10 years ago. Although it is working well, it's ages are starting to show. Rewriting BridgeDB to be compatible with Python 3 while maintaining the status quo seems not only hard but also...BridgeDB, to my knowledge, was developed more than 10 years ago. Although it is working well, it's ages are starting to show. Rewriting BridgeDB to be compatible with Python 3 while maintaining the status quo seems not only hard but also seems like missing a good opportunity to write it again.
I propose rewriting BridgeDB with Django. With Django, BridgeDB codes will be more compact, clean and easy to understand.
**Trac**:
**Username**: moonsikparkhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32276Help BridgeDB see client IP addresses of moat requests2021-08-16T20:11:10ZPhilipp Winterphw@torproject.orgHelp BridgeDB see client IP addresses of moat requestsBridgeDB sees the following HTTP headers for incoming moat requests:
```
Content-Length: ['504']
Via: ['1.1 bridges.torproject.org']
X-Forwarded...BridgeDB sees the following HTTP headers for incoming moat requests:
```
Content-Length: ['504']
Via: ['1.1 bridges.torproject.org']
X-Forwarded-Host: ['bridges.torproject.org']
X-Forwarded-For: ['127.0.0.1']
Connection: ['Keep-Alive']
Host: ['127.0.0.1:3881']
X-Forwarded-Server: ['bridges.torproject.org']
Content-Type: ['application/vnd.api+json']
```
As part of our BridgeDB metrics (legacy/trac#9316) it would be useful to see the IP address of incoming requests.
Here's an IRC conversation in which dcf explains the big picture and proposes a way to fix this issue:
```
dcf1│ phw: there are 2 layers of HTTP in Moat -- one is the CDN-traversal (meek) layer, and one is the end-to-end tunnelled HTTP to the web
server itself.
dcf1│ The CDN will set XFF on the outer meek layer, but not on the inner layer which is what BridgeDB sees (and it couldn't touch the inner
layer because it's HTTPS, which is the whole reason for the complicated proxypass setup).
dcf1│ phw: in other words, XFF is being set on the connection to port 2000, but then that layer is stripped off (only meek-server sees it)
and the tunneled contents go to port 3881.
dcf1│ Even though BridgeDB is an HTTP service, we go to the trouble of tunnelling a whole independent HTTP+TLS stream *inside* the
domain-fronted layer, just to prevent the CDN from tampering with end-to-end traffic.
dcf1│ That's why there are 2 proxypass: the first terminates the meek CDN layer, and the second terminates the tunnelled actual BridgeDB
exchange.
dcf1│ One way to solve this would be yet another shim that understands ExtORPort, parses out the USERADDR, and inserts it into XFF into an
HTTP header. As if the pipeline weren't long enough...
phw│ dcf1: gotcha. i'll create a ticket for this. do you mind if i quote your above explanation?
dcf1│ go for it
phw│ is meek-server exposed to USERADDR? i don't think i follow
dcf1│ meek-server can provide USERADDR to whatever port it forwards to. Currently I'm sure it's set up not to do that, to just forward the
contents without any prefix.
dcf1│ meek-server looks at Meek-IP, X-Forwarded-For, or request source IP address, and uses those to construct a USERADDR, which normally it
would provide to tor on tor's ExtORPort.
dcf1│ The purpose of ExtORPort, as opposed to ordinary ORPort, is to allow passing extra metadata like that, before the actual stream data.
dcf1│ So currently meek-server in Moat is set up to treat the local HTTPS server as its ORPort (not ExtORPort, because Apache wouldn't
understand that).
phw│ dcf1: ah, i understand. thanks for elaborating
dcf1│ Just thought of a research idea: see if any bridges are wrongly exposing their ExtORPort, which would conceivably permit manipulating
statistics.
phw│ we should call this new shim rube-goldberg-machine ;)
dcf1│ Take a subset of bridges, then port-scan them to see if any other port understands ExtORPort-prefixed Tor connections.
dcf1│ Moat is a concrete case where pluggable transports fall short of what you might want them to do. We're trying to "plug" meek-server
into another pipeline, but it's difficult because it's talking to something other than Tor. It's possible, but it quickly turns into a
big pile of shims and adapters.
```Sponsor 30 - Objective 2.1Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32203BridgeDB doesn't create metrics for vanilla bridges2020-06-27T13:42:45ZPhilipp Winterphw@torproject.orgBridgeDB doesn't create metrics for vanilla bridgesThe metrics.py module uses code like the following to weed out invalid transport protocols. Unfortunately, this is also weeding out "vanilla", which results in BridgeDB not counting vanilla bridges:
```
if not isTransportSupported(bridg...The metrics.py module uses code like the following to weed out invalid transport protocols. Unfortunately, this is also weeding out "vanilla", which results in BridgeDB not counting vanilla bridges:
```
if not isTransportSupported(bridgeType):
logging.warning("User requested unsupported transport type %s "
"over HTTPS." % bridgeType)
return
```Sponsor 30 - Objective 2.1Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32134Request new translation and update i18n instructions2021-07-01T17:47:15ZPhilipp Winterphw@torproject.orgRequest new translation and update i18n instructionsWhile implementing our language switcher (legacy/trac#26543), we added a new string, "Language", that requires translations. We should also update our instructions on how to request new translations.While implementing our language switcher (legacy/trac#26543), we added a new string, "Language", that requires translations. We should also update our instructions on how to request new translations.Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32117Understand and document BridgeDB bot scraping attempts2020-10-29T17:40:19ZCecylia BocovichUnderstand and document BridgeDB bot scraping attemptsWe are aware of automated attempts to enumerate bridges in BridgeDB, but lack a more rigorous understanding of the problem.
We have detected bot requests from bridgeDB's web interface and deployed some defences by forbidding requests wi...We are aware of automated attempts to enumerate bridges in BridgeDB, but lack a more rigorous understanding of the problem.
We have detected bot requests from bridgeDB's web interface and deployed some defences by forbidding requests with headers that are commonly associated with bots, and handing out fake bridges to suspected bot requests (legacy/trac#31252), and
We also suspect that these bots are solving our CAPTCHAs more accurately than users (legacy/trac#24607).
After a recent campaign to get more volunteer bridges, we set up an experiment to test the reachability of a subset of these new bridges from a probe site in Beijing and found all new bridges in our sample to be blocked (most were blocked from the very start of the experiment): legacy/trac#31701
This ticket is for documenting bot behaviour and brainstorming ways to detect and analyze the automatic scraping of BridgeDB from censor-owned bots.hanneloresxhanneloresxhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32105bridges@torproject.org don't respond2020-07-14T22:23:15ZTracbridges@torproject.org don't respondI use Gmail to send email to bridges@torproject.org, but it doesn't respond.
**Trac**:
**Username**: mh828I use Gmail to send email to bridges@torproject.org, but it doesn't respond.
**Trac**:
**Username**: mh828Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/32035Only use translations that are >=80% complete2021-05-18T16:07:34ZPhilipp Winterphw@torproject.orgOnly use translations that are >=80% completeThis is a spin-off ticket from legacy/trac#26543. Antonela says:
> I think is good to show just 80% completed languages. If not, the site looks like broken for non-speakers. Let's have emmapeel in this party so we plan an open a call for...This is a spin-off ticket from legacy/trac#26543. Antonela says:
> I think is good to show just 80% completed languages. If not, the site looks like broken for non-speakers. Let's have emmapeel in this party so we plan an open a call for translators just for bridge.tpo and gettor.tpo :)
After a cursory look, babel can tell us the completion of a translation while compiling it:
```
$ python setup.py compile_catalog -l uz
running compile_catalog
25 of 60 messages (41%) translated in bridgedb/i18n/uz/LC_MESSAGES/bridgedb.po
compiling catalog bridgedb/i18n/uz/LC_MESSAGES/bridgedb.po to bridgedb/i18n/uz/LC_MESSAGES/bridgedb.mo
```
Another thing worth considering: 80% completion is not the same as 80% reviewed [as emmapeel explained](https://trac.torproject.org/projects/tor/ticket/26543#comment:23).https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/31967BridgeDB Server uses insecure pseudorandom generator for selecting cached cap...2020-06-27T13:42:46ZTracBridgeDB Server uses insecure pseudorandom generator for selecting cached captchahttps://gitweb.torproject.org/bridgedb.git/tree/bridgedb/captcha.py#n389
From python documentation: The pseudo-random generators of this module (random) should not be used for security purposes.
It should use the secrets module `secret...https://gitweb.torproject.org/bridgedb.git/tree/bridgedb/captcha.py#n389
From python documentation: The pseudo-random generators of this module (random) should not be used for security purposes.
It should use the secrets module `secrets.choice()` or if you plan to keep python2 compatibility `random.SystemRandom.choice()`.
**Trac**:
**Username**: willbarrhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/31936Write usage metrics to disk before terminating2020-06-27T13:42:46ZPhilipp Winterphw@torproject.orgWrite usage metrics to disk before terminatingBridgeDB writes its usage metrics to disk every 24 hours. It does not, however, write the metrics to disk when it's restarted, which means we are losing up to 24 hours worth of usage metrics after each restart. We should make BridgeDB wr...BridgeDB writes its usage metrics to disk every 24 hours. It does not, however, write the metrics to disk when it's restarted, which means we are losing up to 24 hours worth of usage metrics after each restart. We should make BridgeDB write its incomplete metrics to disk before it terminates.
Let's also make sure that this fix plays well with our logrotate cronjob on polyanthum.Sponsor 30 - Objective 2.1https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/31903Update translations and push translation requests to Transifex2021-07-01T17:47:15ZPhilipp Winterphw@torproject.orgUpdate translations and push translation requests to TransifexIt's time to update BridgeDB's existing translations and to push new translation requests because some of BridgeDB's strings have changed in the meanwhile.It's time to update BridgeDB's existing translations and to push new translation requests because some of BridgeDB's strings have changed in the meanwhile.Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/31878Make BridgeDB and bridge authority more resilient2020-11-20T18:46:03ZPhilipp Winterphw@torproject.orgMake BridgeDB and bridge authority more resilientWe should explore options to decentralise BridgeDB and/or our bridge authority.We should explore options to decentralise BridgeDB and/or our bridge authority.Sponsor 30 - Objective 2.4https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/31876Overhaul BridgeDB's documentation and specification2020-10-29T18:39:08ZPhilipp Winterphw@torproject.orgOverhaul BridgeDB's documentation and specificationBridgeDB's documentation is slightly outdated and its [specification](https://gitweb.torproject.org/torspec.git/tree/bridgedb-spec.txt) is severely outdated. It's time to update both.BridgeDB's documentation is slightly outdated and its [specification](https://gitweb.torproject.org/torspec.git/tree/bridgedb-spec.txt) is severely outdated. It's time to update both.Sponsor 30 - Objective 2.4https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/31875BridgeDB should consider a user's location2021-09-09T14:18:39ZPhilipp Winterphw@torproject.orgBridgeDB should consider a user's locationbridges.torproject.org knows the user's physical location and should consider it when returning bridges. For example, if somebody in country X asks for an obfsN bridge but BridgeDB knows that only obfsN+1 works in X (e.g., by consulting ...bridges.torproject.org knows the user's physical location and should consider it when returning bridges. For example, if somebody in country X asks for an obfsN bridge but BridgeDB knows that only obfsN+1 works in X (e.g., by consulting the result of tpo/community/outreach#28531), it should return obfsN+1.
Ideally, we would do this for all of BridgeDB's distribution mechanisms. We could also do it for email – if the user emailed bridges+CC@bridges.torproject.org. As I understand it, we cannot do it for moat because BridgeDB doesn't get to see the user's IP address in this case.Sponsor 30 - Objective 3.3meskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/31873Implement the Salmon bridge distribution mechanism2020-09-02T17:52:43ZPhilipp Winterphw@torproject.orgImplement the Salmon bridge distribution mechanismBridgeDB currently has three bridge distribution mechanisms: Email, HTTPS, and moat. Email is problematic because its interaction mechanism is complicated, not everyone has a Gmail or Riseup address, and it's easy to crawl. HTTPS is prob...BridgeDB currently has three bridge distribution mechanisms: Email, HTTPS, and moat. Email is problematic because its interaction mechanism is complicated, not everyone has a Gmail or Riseup address, and it's easy to crawl. HTTPS is problematic because bridges.torproject.org is blocked in most places that matter and our CAPTCHA is good at keeping out users (legacy/trac#29695) but not so good at keeping out bots (legacy/trac#31252). Moat remains relatively useful because it uses domain fronting but it also relies on a CAPTCHA to fight off bots.
It's time to think about new and/or significantly improved bridge distribution methods. How can we get bridges into the hands of users while making it difficult for adversaries to get them all? How can we make BridgeDB's CAPTCHA more resistant against bots and easier for users?
The Salmon bridge distribution system (first presented in a [PETS'16 paper](https://censorbib.nymity.ch/#Douglas2016a)) is promising. Let's use this issue to build a prototype and fill in the missing pieces to get Salmon deployed.Sponsor 30 - Objective 2.3Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.org