BridgeDB issueshttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues2020-06-27T13:43:01Zhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/17988please provide default bridges to the Whonix project2020-06-27T13:43:01Zadrelanosplease provide default bridges to the Whonix projectThe Whonix project would like to ship default bridges. As suggested by Roger, these should ideally differ from the default bridges shipped with TBB.
The ones that will be hardcoded in the source code. And shipped together with [anon-con...The Whonix project would like to ship default bridges. As suggested by Roger, these should ideally differ from the default bridges shipped with TBB.
The ones that will be hardcoded in the source code. And shipped together with [anon-connections-wizard](https://github.com/Whonix/anon-connection-wizard). (Which is a python rewrite of tor-launcher for the whole system.) ([screenshots](https://www.whonix.org/blog/connection-bridge-wizard))
Ideally, these should not be given out by BridgeDB to other users since these will become semi-public. (Anyone bothering to read the source code will find them.)
Please send them by private [encrypted] mail to adrelanos at riseup dot net.
Keep your time. There likely will be no new release before March 2016.https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/17006different perspective on censorship2020-06-27T13:43:02ZTracdifferent perspective on censorshipbridgedb and gettor are trying to solve a tchnical problem which is a good thing, no doubt but often technical solutions are not the key to changing a society. most of the time it is the way in which technology is brought into peoples li...bridgedb and gettor are trying to solve a tchnical problem which is a good thing, no doubt but often technical solutions are not the key to changing a society. most of the time it is the way in which technology is brought into peoples lives.
i think it is important to look at how censorship works and how it is integrates with peoples lives:
prevent people from spreading and finding specific information.
One average way someone uses tor could be like this:
-i feel like the goverment is hiding something from me.
-i should try to find a way to get around this.
-i found tons of different tools to do that but what is the best?
-i gonna use tor
-but the goverment makes it difficult for me to get it.
-i have to try to find a way to get it working.
-after a while i finally got it working
-i now have full access to the whole internet...
-now where are the interesting things the goverment is hiding?
-where should i start searching?
-...hmm this thing is nice to have but without aquiring further knowlege its probably not of much use for the average user...
-i probably should look into this some other day.
the problem here is that it requires initiative several times.
except for porn people usually do not show this behavior. thats why facebook and tv are so popular. most humans are passive animals and let others decide the boundaries of ther lives. so there are 2 ways of reaching people: attatch to something within the boundaries or change the boundaries. the best way is to start with the first and continue with the second. with other words we should find a way to make tor a thing.
but starting a movement is hard and usually not going to work. it has to evolve naturally. societies have a gigantic amount of trial and error potential. if a creative environment is being provided a crowd can come up with amazing solutions. creativity works the best the way of expression is universal but some not too tight and not to loose boundaries exist in one or few dimensions.
so what bridgedb gettor and other related projects should do is to find tools that can provide such properities.
on idea i had for example was to make a website or a button in torbrowser that generates a qr code that a mirror for torbrowser, a set of bridges and an url inside. that way all is in one place. the information and the tool to access it. it can be share with friends on a smartphone, stickers could be made of it or shared in any channel that is difficult for censors to control
thats only on idea. there are probably way more different things.
**Trac**:
**Username**: elypterIsis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/15457Separate bridgedb.txrecaptcha into another package2021-09-09T14:29:09ZIsis LovecruftSeparate bridgedb.txrecaptcha into another packageWe should separate txrecaptcha into its own package so that others can use it, and also to reduce the amount of code that is non-BridgeDB-specific which resides in BridgeDB.We should separate txrecaptcha into its own package so that others can use it, and also to reduce the amount of code that is non-BridgeDB-specific which resides in BridgeDB.https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/14453Implement statistics gathering for number of Bridges-per-Transport in BridgeDB2020-06-27T13:43:06ZIsis LovecruftImplement statistics gathering for number of Bridges-per-Transport in BridgeDBAs part of the [SponsorS PT work](https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorS/PluggableTransports), we promised a way to gather statistics on the number of bridges per transport.
The proposal states this is a tas...As part of the [SponsorS PT work](https://trac.torproject.org/projects/tor/wiki/org/sponsors/SponsorS/PluggableTransports), we promised a way to gather statistics on the number of bridges per transport.
The proposal states this is a task for Metrics. However, it's possible to do this on the BridgeDB side. In fact, it would help BridgeDB in the future to determine how to better allocate bridges to its Distributors (and help the Distributors hand them out to users in smarter ways).
Technically, BridgeDB already sort-of has data on the number of Bridges-per-Transport… or, rather, when a client requests a certain type of bridge from a certain Distributor (e.g. "give me an IPv4 obfs3 bridge from the HTTPS Distributor"), BridgeDB creates (or retrieves from a cache) a "filtered" subhashring containing only Bridges which fit the client's request. BridgeDB even logs the number of Bridges in these subhashrings in its DEBUG and INFO logs:
```
22:19:16 INFO L1361:Bridges.addRing() Bridges inserted into HTTPS-Transpo subring: 235
22:19:16 DEBUG L75:Dist.getNumBridgesPerA() Returning 3 bridges from ring of len: 235
```
The problem with using those numbers for statistics is that BridgeDB's Distributors may have multiple adjacent subhashrings, usually about 5. So, in the above case, there's roughly something like 1175=5*235 obfs3 bridges in the HTTPS Distributor. (These numbers aren't from the real deployed BridgeDB, by the way.)
---------
A better way to do this would be to provide a database query (as part of legacy/trac#12031) which counts the number of Bridges which claim to offer a PT. An example mechanism for doing this in Redis would be to keep a hash (i.e. using [HSET](http://redis.io/commands/hset) or `HINCRBY`) of Bridges which have any PTs, where the keys are the Bridge fingerprints, add a field for each type of PT, and then (if not using `HINCRBY`) store `IP:PORT[,IP:PORT[,IP:PORT[…]]]`, for example:
```
redis> HSET 26F6A7570E0F655DFDD054E79ACBB127112C2D7B obfs4 "4.4.4.4:4444,5.5.5.5:5555"
```
With that scheme, a new `HSET` would be necessary each time the `@type bridge-extrainfo` descriptors are parsed, but this only has time complexity O(1).
Some considerations / additional query parameters:
* For these statistics, should we only count Bridges with the Running flag? Or only if the OONI machine says the PT is reachable?
* What sanitisations should be done on these numbers? Should we round them? Or provide a scale, i.e. "between 1000-5000 obfs4 bridges"?
* Do we want only the _Bridges_ with a given PT? Or do we want the _number of instances_ of a given PT (e.g. if a Bridge has multiple obfs3 instances)?Sponsor 30 - Objective 2.1https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/12806Create Debian package for BridgeDB2020-06-27T13:43:08ZIsis LovecruftCreate Debian package for BridgeDBSo that other organisations who wish to run their own BridgeDB which distributes their own bridges can do so. It's unclear right now if this is possible, or even necessary, completing legacy/trac#12805 may suffice.So that other organisations who wish to run their own BridgeDB which distributes their own bridges can do so. It's unclear right now if this is possible, or even necessary, completing legacy/trac#12805 may suffice.https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/12805Package BridgeDB on PyPI2020-06-27T13:43:08ZIsis LovecruftPackage BridgeDB on PyPIIn preparation for creating a Debian package out of it, in the hopes that some day other organisations will be capable of running their own BridgeDB's with their own bridges for distribution.In preparation for creating a Debian package out of it, in the hopes that some day other organisations will be capable of running their own BridgeDB's with their own bridges for distribution.Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/12547Get analysed data from bridge reachability tests to tor-devs2021-09-09T14:26:41ZArturo FilastòGet analysed data from bridge reachability tests to tor-devsThis means setting up a web server on the post processing machine (the one running the collector) with some access control so that tor devs can read the reports.This means setting up a web server on the post processing machine (the one running the collector) with some access control so that tor devs can read the reports.https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/12545Move collector of bridge reachability measurements to bridge db2020-06-27T13:43:11ZArturo FilastòMove collector of bridge reachability measurements to bridge dbThis means setting up oonib on a box run by the bridge db team.
For documentation on how to setup oonib check out: https://gitweb.torproject.org/ooni/oonib.git/blob/HEAD:/README.mdThis means setting up oonib on a box run by the bridge db team.
For documentation on how to setup oonib check out: https://gitweb.torproject.org/ooni/oonib.git/blob/HEAD:/README.mdMatthew FinkelMatthew Finkelhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/10723Create a staging instance for BridgeDB2020-06-27T13:43:18ZIsis LovecruftCreate a staging instance for BridgeDBBridgeDB needs a staging instance for checking that a new release works as expected when deployed.
This staging instance should have it's own setup/deployment scripts (probably in the bridgedb-admin repo). Ideally it should be able to t...BridgeDB needs a staging instance for checking that a new release works as expected when deployed.
This staging instance should have it's own setup/deployment scripts (probably in the bridgedb-admin repo). Ideally it should be able to take some sort of argument for which release/commit/branch to install and fire up as the staging instance, and other than that it should be as much as possible like the normal staging scripts.
Anything else I'm missing? Any suggestions?Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9865Create a test harness for BridgeDB2020-06-27T13:43:20ZIsis LovecruftCreate a test harness for BridgeDBBridgeDB's tests are currently stored in lib/BridgeDB/Tests.py and use the standard library's `unittest` module. Normally, Twisted code is tested with `twisted.trial`, for the reason that if something returns a `twisted.internet.defer.De...BridgeDB's tests are currently stored in lib/BridgeDB/Tests.py and use the standard library's `unittest` module. Normally, Twisted code is tested with `twisted.trial`, for the reason that if something returns a `twisted.internet.defer.Deferred` the stdlib `unittest` module doesn't know to wait for the result (nor does it understand how to clear the reactor).
Some things which should be done:
* We need a test runner which can pass on commands (i.e. doing "python setup.py test" would actually run "trial coverage [args] [tests/test directory]" and pass the arguments and test cases to coverage if need be. See tahoe-lafs's code for an example) and environment variables (i.e. `PYTHONPATH`).
* `twisted.trial` has a conversion utility class for translating test results from stdlib `unittest` into something which `trial`'s reporter can use. I've never used it, but either some sort of conversion should be done of the old tests, or they should be rewritten.
* Find a way to write integration tests for the web interface.[Windmill](http://www.getwindmill.com/) ([more docs](https://github.com/windmill/windmill/wiki/IDE)) and [twill](http://twill.idyll.org/python-api.html) look promising. Windmill probably wouldn't run very nicely on a headless server, however, as it drives a browser.
* Find a way to write tests for the email responder.
If we get all this running, we can setup Travis-CI and get automated continuous integration, which should help prevent bugs like legacy/trac#9626 and legacy/trac#9156.
Useful Resources:
http://pycheesecake.org/wiki/PythonTestingToolsTaxonomyIsis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9615Copy current BridgeDB deployment scripts into admin/bridgedb-admin repo2020-06-27T13:43:20ZIsis LovecruftCopy current BridgeDB deployment scripts into admin/bridgedb-admin repoThe current deployment scripts should be put into the new repo (ticket legacy/trac#9595).
Also, the current layout of the home directory, stat outputs for files, any logrotate configs, and perhaps a copy of the cronjob should be recorde...The current deployment scripts should be put into the new repo (ticket legacy/trac#9595).
Also, the current layout of the home directory, stat outputs for files, any logrotate configs, and perhaps a copy of the cronjob should be recorded somewhere in there as well.
Obviously, everything will need to be checked that it does not contain any sensitive material.Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9462BridgeDB networkstatus descriptor parsers need refactoring2020-06-27T13:43:21ZIsis LovecruftBridgeDB networkstatus descriptor parsers need refactoringSome of this stuff I am not sure if it was part of some super old proposal for some descriptor type. Other stuff is just janky. For example, [one of the bugs I just found](https://gitweb.torproject.org/user/isis/bridgedb.git/blob/HEAD:/l...Some of this stuff I am not sure if it was part of some super old proposal for some descriptor type. Other stuff is just janky. For example, [one of the bugs I just found](https://gitweb.torproject.org/user/isis/bridgedb.git/blob/HEAD:/lib/bridgedb/Bridges.py#l469), on L469 in lib/bridgedb/Bridges.py, wouldn't ever get hit because to my knowledge BridgeDB has never handed out fingerprints. But if it were to, with those lines, it would hand out PT bridges that look like this:
```
bridge obfs3 1.2.3.4:5555 keyid=0123456789abcdef0123456789abcdef01234567
```
with the part after `keyid=` being the bridge's fingerprint.
Another thing which I am currently looking into is BridgeDB's parsing of the networkstatus files, it seems BridgeDB expects there to _always_ be an `a` line with the ORaddress, whereas this is only the case if the bridge has at least one IPv6 address.
BridgeDB also seems to think that a bridge/relay can't have more than 8 ORaddress:ORport pairs. I remember this discussed years ago, though I thought it was 16, and I don't recall if it was ever implemented in little-t tor.
In short, there is quite a bit of jankiness in BridgeDB's descriptor parsing. I am going to continue fixing things and referencing dir-spec.txt when there is a discrepancy, and just calling BridgeDB out when there appears to be no sane reason for something referenced in any of torspec.Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9443Generate and secure pgp keys for bridges.tpo2020-06-27T13:43:21ZMatthew FinkelGenerate and secure pgp keys for bridges.tpoWe need to start signing the emails we send from bridges.tpo, but we need keys to do this. This means we need to be able to store them in a safe way, too; preferably the long-term keys will be stored in an offline hardware chip of some s...We need to start signing the emails we send from bridges.tpo, but we need keys to do this. This means we need to be able to store them in a safe way, too; preferably the long-term keys will be stored in an offline hardware chip of some sort, subkey(s) will be online. Who will generate these, who will control this chip and where will it be stored?
Plus any other important questions I missed.Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9425Create and document a better BridgeDB (re)deployment strategy2020-06-27T13:43:21ZIsis LovecruftCreate and document a better BridgeDB (re)deployment strategyBridgeDB's current deployment strategy is a little bit crazy.
It would be nice to have automatic deployments, i.e. if a signed git tag of a version number is pushed, and that tag is signed by one of a specific set of keys, it automatica...BridgeDB's current deployment strategy is a little bit crazy.
It would be nice to have automatic deployments, i.e. if a signed git tag of a version number is pushed, and that tag is signed by one of a specific set of keys, it automatically redeploys itself. This might be a tiny bit tricky to set up, and I'm not sure what the opinions of phobos, arma, weasel, et al. are on this.
For the present, I propose the following:
1) **Versioning**: Using [versioneer](https://github.com/warner/python-versioneer?source=cc) to automatically create version strings from git tags, and if the current deployment is not directly from a tag, then it automatically creates the version string from the most recent tag + short commit ID.
2) **Patch Management**: I also propose that some sort of patch management system be used for applying tpo-specific infrastructure changes, like changing the install path, bound IP addrs/ports in the bridgedb.conf file, etc. I've used quilt and it works well; if someone knows of something that is better than quilt for some reason, let me know. Otherwise, I would go with using quilt in a patches/ directory that is a separate git repository.
3) **Cleanup**: There are a lot of files/directories lying around ponticum.tpo, including, iirc, three copies of bridgedb.git. Some of these files appear to not be used anymore, others I have no idea if they are/were used for deployment, or are used for some metrics tasks, or what.Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9332Implement whitelisting of (email_address, gpg_key_id) pairs for encrypted, au...2021-09-09T14:20:45ZGeorge KadianakisImplement whitelisting of (email_address, gpg_key_id) pairs for encrypted, automated email bridge distributionRoger told me that BridgeDB used to send bridges to a list of emails. It got those bridges from the reserved pool, and sent some of them to the members of the mailing list every so often.
This feature seems to be disabled now (for some ...Roger told me that BridgeDB used to send bridges to a list of emails. It got those bridges from the reserved pool, and sent some of them to the members of the mailing list every so often.
This feature seems to be disabled now (for some reason), but it might be a good idea to revive it.https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9316BridgeDB should export statistics2020-06-27T13:43:22ZGeorge KadianakisBridgeDB should export statisticsBridgeDB should export statistics on its usage. Stuff like distributor usage, number of clients served, etc.
Ticket legacy/trac#19332 tracks our follow-up task: the inclusion of BridgeDB's metrics into CollecTor.BridgeDB should export statistics on its usage. Stuff like distributor usage, number of clients served, etc.
Ticket legacy/trac#19332 tracks our follow-up task: the inclusion of BridgeDB's metrics into CollecTor.Sponsor 30 - Objective 2.1Philipp Winterphw@torproject.orgPhilipp Winterphw@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9199Rethink the logging of BridgeDB2020-06-27T13:43:23ZGeorge KadianakisRethink the logging of BridgeDBNow that we are playing around with BridgeDB, it's probably a good idea to also improve its logging.
For example, BridgeDB currently doesn't have any debug logs during descriptor parsing (which made it harder to find the root of legacy/...Now that we are playing around with BridgeDB, it's probably a good idea to also improve its logging.
For example, BridgeDB currently doesn't have any debug logs during descriptor parsing (which made it harder to find the root of legacy/trac#9174).
Also, we should make sure that BridgeDB can scrub client's IPs from the logs (even for debug logs). Example of offending logs:
Valid recaptcha from <IP>...
and
[DEBUG] getBridgesForIP(<IP>, ...)
Also, maybe we should consider using Tor's extended nicknames to improve log greppability. Extended nicknames combine the identity digest and the nickname of a router in the same line. Example:
`$AAAAAAAA01234AAAAAAAAAAAAAAAAAAAAAAAAAAA=fred` .
Check Tor's function `node_get_verbose_nickname()` and https://gitweb.torproject.org/torspec.git/blob/HEAD:/control-spec.txt#l152
We should also identify other information that the BridgeDB operator would be interested in and log them (maybe even write a heartbeat log with statistics).Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9157Persian and Arabic should be right aligned on bridges.tpo2020-06-27T13:43:23ZRuna SandvikPersian and Arabic should be right aligned on bridges.tpoPersian and Arabic should be right aligned on bridges.torproject.org.Persian and Arabic should be right aligned on bridges.torproject.org.Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9156BridgeDB: Users try to add obfsbridges to their normal TBB2020-06-27T13:43:23ZGeorge KadianakisBridgeDB: Users try to add obfsbridges to their normal TBBPhoul told me today that some users get bridges from the new BridgeDB page, and then try to add them to a normal Tor Browser Bundle (that doesn't understand PTs).
This means that those users completely ignored 'Step 1: Get the Pluggable...Phoul told me today that some users get bridges from the new BridgeDB page, and then try to add them to a normal Tor Browser Bundle (that doesn't understand PTs).
This means that those users completely ignored 'Step 1: Get the Pluggable Transports Tor Browser Bundle'. Can we help those users out somehow?
We could **force** users to download the bundle by making BridgeDB a wizard that actually requires you to do step 1 before proceeding to step 2. I'm not sure if this is a good idea though. It might infuriate users who know what they are doing.Isis LovecruftIsis Lovecrufthttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/9127Users can't ask for ipv6 bridges with the new bridgedb interface2020-06-27T13:43:23ZGeorge KadianakisUsers can't ask for ipv6 bridges with the new bridgedb interfaceThe new BridgeDB interface doesn't allow users to ask for IPv6 bridges. Apparently, the ipv6=true GET parameter trick still works, but we shouldn't rely on users guessing it.
We should think how the "Give me IPv6 bridges" button should ...The new BridgeDB interface doesn't allow users to ask for IPv6 bridges. Apparently, the ipv6=true GET parameter trick still works, but we shouldn't rely on users guessing it.
We should think how the "Give me IPv6 bridges" button should be integrated in the current interface.
For example, if we wanted to keep the current KISS interface, maybe we shouldn't add a new button, and instead we should just return IPv6 bridges like they are normal bridges (and expect the latest PTTBB and the user's OS to support IPv6).Isis LovecruftIsis Lovecruft