Anti-censorship issueshttps://gitlab.torproject.org/groups/tpo/anti-censorship/-/issues2022-03-31T10:20:33Zhttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/100in moat's circumvention defaults you can get the same bridge twice2022-03-31T10:20:33Zmeskiomeskio@torproject.orgin moat's circumvention defaults you can get the same bridge twiceBridges are selected randomly and is not checking that they are different.Bridges are selected randomly and is not checking that they are different.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/99distributors should wait a delay to retry the connection to the backend2022-06-02T10:08:28Zmeskiomeskio@torproject.orgdistributors should wait a delay to retry the connection to the backendEach retry produces an entry in the log, a missconfigured service was retrying so frequently that has produced 10GB of logs and fill the disk.Each retry produces an entry in the log, a missconfigured service was retrying so frequently that has produced 10GB of logs and fill the disk.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/97shorten the period to get fresh bridges in circumvention settings2023-09-11T15:36:19Zmeskiomeskio@torproject.orgshorten the period to get fresh bridges in circumvention settingsAt !30 I implemented (from the [doc I wrote](https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/blob/0d4729347c11bb476fcb41eaff47d50241a7567a/doc/moat.md)):
> The resources provided /circumvention/settings and /circumvention/defa...At !30 I implemented (from the [doc I wrote](https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/blob/0d4729347c11bb476fcb41eaff47d50241a7567a/doc/moat.md)):
> The resources provided /circumvention/settings and /circumvention/defaults use a combination of two mechanisms to make it harder for attackers to list all the bridges.
>
> Resources are group so each resource will only be distributed in a certain time period (rotation_period_hours), and will not be distributed again until a number of periods has past (num_periods). If rotation_period_hours=24 and num_periods=30 resources will be divided in 30 groups and each group will be distributed during one day and so a single resource will not be distributed again 30 days has passed.
>
> The IP address of the requester will be used so over the same rotation period every IP coming from the same subnet will get the same resources on each request.
I have some doubts about that mechanism. I'm thinking on setting `rotation_period_hours=24`. So if the bridges that a user gets don't work they need to wait for a day to be able to request new bridges, might that be too long? Might we want to add another rotation period for the IP subnet, something like the current MOAT_ROTATION_PERIOD=3h, so from the same bridge group a single subnet can get different bridges each 3 hours. Or will that just complicate the mechanism without adding much?
I'm using the same subnet definition that [BridgeDB](https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/blob/9f739738fe1eed32933a9fa0165b0835907280e2/bridgedb/distributors/https/distributor.py#L110), /16 for IPv4 and /32 for IPv6. Specifically for IPv4 I guess many countries will not have so many subnets, which means that most users in a country requesting bridges in a single rotation period will get to few bridges. Is that concern real? The second rotation period would help here as well, or we could reduce the subnets (to /24 for IPv4).
(from https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/79#note_2788654)Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/96only distribute functional bridges tested by bridgestrap2023-05-10T17:46:44Zmeskiomeskio@torproject.orgonly distribute functional bridges tested by bridgestrapWe are testing bridges with bridgestrap and using that results for metrics and to provide feedback to bridge operators on the status of their bridges. But having a fast look into the code doesn't look like we are using that results to di...We are testing bridges with bridgestrap and using that results for metrics and to provide feedback to bridge operators on the status of their bridges. But having a fast look into the code doesn't look like we are using that results to distribute only functional bridges.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/95telegram l10n support2023-08-23T18:59:04Zmeskiomeskio@torproject.orgtelegram l10n supportIs l10n support possible? how does this work?Is l10n support possible? how does this work?Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40046add description of settings and telegram distribution mechanisms2022-05-04T11:28:58Zmeskiomeskio@torproject.orgadd description of settings and telegram distribution mechanismshttps://bridges.torproject.org/info is missing a description of the new distribution mechanisms, and the metrics relay search points to it.https://bridges.torproject.org/info is missing a description of the new distribution mechanisms, and the metrics relay search points to it.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/93are we acknoleging the `bridge-distribution-request`?2022-05-02T07:14:20Zmeskiomeskio@torproject.orgare we acknoleging the `bridge-distribution-request`?In the assignments file there are bridges assigned to distributors different than the one set in their `bridge-distribution-request`, or assigned to distributors when they have it set to `none`. Are we distributing them or is a problem i...In the assignments file there are bridges assigned to distributors different than the one set in their `bridge-distribution-request`, or assigned to distributors when they have it set to `none`. Are we distributing them or is a problem in the assignments file?Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/90review the concurrent usage of the BackendContext2022-02-25T13:44:42Zmeskiomeskio@torproject.orgreview the concurrent usage of the BackendContext@shelikhoo has raised the doubt if we are using safely BackendContext between goroutines: https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/merge_requests/27#note_2780229
Let's have a look to it.@shelikhoo has raised the doubt if we are using safely BackendContext between goroutines: https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/merge_requests/27#note_2780229
Let's have a look to it.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/65S96 dynamic IP obfs4 bridge performance insufficiency2023-04-10T17:37:21ZshelikhooS96 dynamic IP obfs4 bridge performance insufficiencyThis issue describes a potential usability issue with S96 dynamic IP address obfs4 bridges. The network connectivity between these bridges hosted at OVH and Tor's China vantage has an insufficient performance level to get Tor bootstrap c...This issue describes a potential usability issue with S96 dynamic IP address obfs4 bridges. The network connectivity between these bridges hosted at OVH and Tor's China vantage has an insufficient performance level to get Tor bootstrap consistently.
Tor like any network application assumes a certain level of network performance to operate [correctly](https://gitlab.torproject.org/tpo/core/tor/-/issues/16844). However, the dynamic IP obfs4 bridge currently set up have a download speed of 5-20 KByte/s during some time of the day. This speed makes its end-user experience inconsistent, and sometimes stop working(This is observed by both @cohosh and me when the bridge is unable to finish bootstrap and progress stuck at 50%).
Additional diagnose(see [below](https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/65#note_2796296)) show ping to Sr2CobaltPluto bridge provided by @irl have a packet loss of 8%. For an ordinary TCP stack, this packet loss level will almost always result in a slow transfer speed.
The root cause of this issue is the TCP stack used is not designed to sufficiently compensate for packet loss; network link to OVH instance is throttled(intentional or not) with 1. transit via congested [AS4134](https://bgp.he.net/AS4134) CHINANET backbone network (as presumably a deprioritized traffic), 2. traffic between instance located Europe and vantage point took a detour at the US(see traceroute in the link above).
~~This issue is marked as confidential to prevent the IP address of our vantage point become public.~~
(traceroute info removed to make this ticket public.)Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetshelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/88bridgestrap still scans bridges but won't admit that any of them exist2022-02-21T17:00:32ZRoger Dingledinebridgestrap still scans bridges but won't admit that any of them existI see in bridgestrap's logs that it is still scanning bridges, e.g.
```
2022/02/08 19:56:46 Tested 25 bridges: 24 (96.0%) functional; 1 (4.0%) dysfunctional.
```
and meskio says that "the graphana dashboard for bridgestrap looks good, m...I see in bridgestrap's logs that it is still scanning bridges, e.g.
```
2022/02/08 19:56:46 Tested 25 bridges: 24 (96.0%) functional; 1 (4.0%) dysfunctional.
```
and meskio says that "the graphana dashboard for bridgestrap looks good, most bridges being functional"
but whenever I try to test any bridge via the web interface with its fingerprint, including fingerprints that I just watched bridgestrap successfully reach in its logs, it tells me
```
no resources for the given id
```
That is, it looks like the backend testing is going fine, but somehow the frontend has become disconnected from it.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/87Single source of truth for builtin bridges2022-03-11T17:38:57Zmeskiomeskio@torproject.orgSingle source of truth for builtin bridgesThe current circumvention API (!15) provides bridges manually configured in rdsys config. So each time we change the default bridges we need to do it in two places: torbrowser & rdsys.
Should rdsys get the bridges from the [torbrowser r...The current circumvention API (!15) provides bridges manually configured in rdsys config. So each time we change the default bridges we need to do it in two places: torbrowser & rdsys.
Should rdsys get the bridges from the [torbrowser repo](https://gitweb.torproject.org/builders/tor-browser-build.git/tree/projects/tor-browser/Bundle-Data/PTConfigs/bridge_prefs.js)? Do we have any better source of builtin bridges? Do we prefer to maintain them manually configured in rdsys?Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/86gettor - Create alias "macos"2022-06-23T14:43:09ZGusgettor - Create alias "macos"We got this feedback:
> It seems that the instruction (https://support.torproject.org/censorship/gettor-2/) lists the macOS as macOS, while the email service requires "osx" string to be present
> for the valid instructions to be returne...We got this feedback:
> It seems that the instruction (https://support.torproject.org/censorship/gettor-2/) lists the macOS as macOS, while the email service requires "osx" string to be present
> for the valid instructions to be returned.
>
> Consider making an alias from "macos" to "osx" so that the users won't be confused.
-----
If that is not possible, I will fix the main support documentation.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40045Circumvention settings fallback defautls2022-04-14T08:38:49Zmeskiomeskio@torproject.orgCircumvention settings fallback defautls> I think it would be a good idea for rdsys to provide what it considers a good 'fallback' set of settings for users to try in the event that they are blocked but do not live in one of the countries we have settings for. For now I assume...> I think it would be a good idea for rdsys to provide what it considers a good 'fallback' set of settings for users to try in the event that they are blocked but do not live in one of the countries we have settings for. For now I assume using an obfs4 builtin bridges is the best idea, but that will surely change at some point in the (hopefully distant) future. So an endpoint like circumvention/reasonable_defaults or whatever which returns an array of settings objects in the same format as the circumvention/settings API would be lovely.
Related to #40043.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40095Add load balancing to bridge2022-10-17T05:15:38ZDavid Fifielddcf@torproject.orgAdd load balancing to bridgeWe rehearsed a load-balanced bridge installation in #40091. Now let's do it for the production bridge. To reduce risk, we plan to do a staged upgrade using a secondary bridge.
We still do not have a permanent solution to the [onion key ...We rehearsed a load-balanced bridge installation in #40091. Now let's do it for the production bridge. To reduce risk, we plan to do a staged upgrade using a secondary bridge.
We still do not have a permanent solution to the [onion key rotation issue](https://forum.torproject.net/t/tor-relays-how-to-reduce-tor-cpu-load-on-a-single-bridge/1483/11). The current plan is to periodically reset [LastRotatedOnionKey](https://gitweb.torproject.org/tor.git/tree/doc/state-contents.txt?h=tor-0.4.6.9#n82) in the state file of all tor instances.
- [x] ask sysadmin team to reduce TTL for snowflake.torproject.net to 60 seconds tpo/tpa/team#40594
- [x] copy user accounts to staging bridge https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40091#note_2768855
- [x] install new staging bridge ([installation guide](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40091#note_2770360))
- [x] refresh the LastRotatedOnionKey line in the state file of the production bridge and restart tor
- [x] back up identity and onion keys from production bridge
- [x] copy identity and onion keys from production bridge to the staging bridge
- [x] copy HTTPS TLS keys and certificates from the production bridge to the staging bridge
- [x] test HTTPS of staging bridge using `curl --connect-to`
- [x] test tor bootstrap on staging bridge using local broker and proxy, and temporary domain name https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40091#note_2770360
- [x] switch DNS for snowflake.torproject.net to point to staging bridge tpo/tpa/team#40598
- [x] monitor for a day, and be ready to switch DNS back to production if connections fail on the staging bridge
- [x] disable and mask tor@default instance on production bridge
- [x] install load balancing configuration on production bridge [installation guide](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40091#note_2770360)
- [x] test HTTPS of production bridge using `curl --connect-to`
- [x] test tor bootstrap on production bridge using local broker and proxy, and temporary domain name https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40091#note_2770360
- [x] switch DNS for snowflake.torproject.net to point back to production bridge tpo/tpa/team#40602
- [x] monitor for 2 days, and be ready to switch DNS back to staging if connections fail on the production bridge
- [x] ask sysadmin team to restore TTL for snowflake.torproject.net to normal tpo/tpa/team#40595
- [x] shut down staging bridge
- [x] post new instructions to [Snowflake Bridge Installation Guide](https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Survival-Guides/Snowflake-Bridge-Installation-Guide) and [Snowflake Bridge Survival Guide](https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Survival-Guides/Snowflake-Bridge-Survival-Guide)
References
* #40091
* https://forum.torproject.net/t/tor-relays-how-to-reduce-tor-cpu-load-on-a-single-bridge/1483
* http://meetbot.debian.net/tor-meeting/2022/tor-meeting.2022-01-20-15.59.log.html#l-60Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & TibetDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40094Disable standalone proxies on bridge2022-01-19T07:40:48ZDavid Fifielddcf@torproject.orgDisable standalone proxies on bridgeWe want to increase the tor capacity of the snowflake bridge. We should disable the two standalone proxies we run on that host, in order to open up some more CPU headroom. Cf. #40091.We want to increase the tor capacity of the snowflake bridge. We should disable the two standalone proxies we run on that host, in order to open up some more CPU headroom. Cf. #40091.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & TibetDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/84Don't distribute bridges with 'none' bridge-distribution-request2022-01-18T16:35:58Zmeskiomeskio@torproject.orgDon't distribute bridges with 'none' bridge-distribution-requestWe don't want to distribute private bridges. How is currently bridgedb handling that? is just ignoring them?We don't want to distribute private bridges. How is currently bridgedb handling that? is just ignoring them?Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/83Circumvention map API should provide the list of countries that needs circumv...2022-02-10T19:57:48Zmeskiomeskio@torproject.orgCircumvention map API should provide the list of countries that needs circumventionLet's add an extra API endpoint that just provides the list of countries that have settings for: `/circumvention/countries`.Let's add an extra API endpoint that just provides the list of countries that have settings for: `/circumvention/countries`.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/82Distributor proportions for deployment2022-03-28T15:17:59Zmeskiomeskio@torproject.orgDistributor proportions for deploymentWe are working on getting rdsys deployed alongside with bridgedb (#12). What distributors we want to configure and with what proportions?
Currently bridgedb [proportions are](https://gitlab.torproject.org/tpo/anti-censorship/bridgedb-ad...We are working on getting rdsys deployed alongside with bridgedb (#12). What distributors we want to configure and with what proportions?
Currently bridgedb [proportions are](https://gitlab.torproject.org/tpo/anti-censorship/bridgedb-admin/-/blob/main/etc/bridgedb.conf#L817):
```
MOAT_SHARE = 20
HTTPS_SHARE = 10
EMAIL_SHARE = 2
RESERVED_SHARE = 2
```
Beside this 4 pools we might want to allocate already a pool for the telegram bot (#77) and for the circumvention settings (bridgedb#40025).
The current usage of each distributor looks like:
![bridgedb-distributor-2021-10-15-2022-01-13](/uploads/8ef66c15ed1a4893f851b9e720f30d6e/bridgedb-distributor-2021-10-15-2022-01-13.png)
How about this configuration for rdsys?
```json
"distribution_proportions": {
"moat": 20,
"moat_settings": 20,
"https": 10,
"email": 2,
"telegram": 2,
"reserved": 2
}
```
(we need to figure out a better name for the circumvention settings distributor)
The current ratio between moat and https is bigger than 2 times (3-6 times), but adding moat_settings I expect the settings to be used at least as much as moat directly probably more.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40043Document the new censorship-circumvention APIs2022-04-20T11:36:40ZrichardDocument the new censorship-circumvention APIsWe need a centralized place we can point developers to for directions on consuming the new (and old) Moat APIs.We need a centralized place we can point developers to for directions on consuming the new (and old) Moat APIs.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40042filtered ring names are not readable2022-01-13T15:00:10Zmeskiomeskio@torproject.orgfiltered ring names are not readableSince !23 got merged ring names look like `frozenset({<function bySubring.<locals>._bySubring at 0x7ba09d305c10>, <function byIPv.<locals>._byIPv at 0x7ba0a63d1dc0>})`. Before the merge they were the ring name: moat, https, email, ...
S...Since !23 got merged ring names look like `frozenset({<function bySubring.<locals>._bySubring at 0x7ba09d305c10>, <function byIPv.<locals>._byIPv at 0x7ba0a63d1dc0>})`. Before the merge they were the ring name: moat, https, email, ...
Somewhere the name is being messed up.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetmeskiomeskio@torproject.orgmeskiomeskio@torproject.org