Snowflake issueshttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues2022-05-06T00:48:32Zhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40132Discussion: libp2p expansion.2022-05-06T00:48:32ZcheakoDiscussion: libp2p expansion.I've always wanted to spin up my own p2p networks. Perhaps you don't agree that anyone should be able to share in one network with islands for each client program. I believe tor is mutually beneficial to this goal. I've studied arti a...I've always wanted to spin up my own p2p networks. Perhaps you don't agree that anyone should be able to share in one network with islands for each client program. I believe tor is mutually beneficial to this goal. I've studied arti and am sad that it doesn't yet support the full for protocol. It looks like there just may be enough there to leach off of tor bridges. That's not the kind of good-natured application I'm interested in.
The libp2p supporters seem uninterested in anything other than a software as a service model. They are uninterested in any p2p network that could stand on its own.
I've looked at snowflake and see that it doesn't currently meet my needs, but it's close.
The kind of application I would build is for playing cards/chess, message boards and sharing doom demo files/high scores. Users would try and be connected all the time, but some would connect for a few hours every day.
The problem tor solves is nat traversal. It would be nice if bootstrapping was also available. At the start of each network there is only one or two nodes. libp2p solves this problem with a "global" DHT, with each program supporting this DHT even if there is only one client connecting there will be nodes running a completely different program but they will serve out the DHT with data from a host that may be offline ATM.
I believe snowflake could be improved in two main ways. 1. Is for the broker to be generic enough for any application to use it. 2. Is for client to proxy connections to be build on top of libp2p.
I envision a future where libp2p clients expand tor DHT and network in exchange for being a rally point for beginner p2p networks.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40131Unix socket support.2023-01-05T17:20:17ZcheakoUnix socket support.I usually want to have nginx reverse proxy over Unix sockets. Otherwise, there are strange fw rules for "random" port numbers.
For both the websocket to tor server and the broker.I usually want to have nginx reverse proxy over Unix sockets. Otherwise, there are strange fw rules for "random" port numbers.
For both the websocket to tor server and the broker.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40129Distributed Snowflake Server Support2024-02-28T14:02:57ZshelikhooDistributed Snowflake Server SupportWe are currently working on making Snowflake more [distrubuted](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/28651#note_2787394). And this ticket will be used to track the progress of implemen...We are currently working on making Snowflake more [distrubuted](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/28651#note_2787394). And this ticket will be used to track the progress of implementing the proposal made in the respective ticket.
- [x] Implementing Client Bridge Fingerprint Indication [MR](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/81)
- [x] Implementing Bridge List Definition Parser
- [x] Implementing Relay Host Name Pattern Matcher
- [x] Implementing Proxy(forwarder) Distributed Snowflake Server Support Indication Messaging Format Support
- [x] Implementing Broker Relay URL Indication to Proxy(forwarder)
- [x] Implementing Proxy(forwarder) Custom Relay URL Support
- [x] Implementing Proxy(forwarder) Custom Relay URL Hostname Pattern Matching Guard
- [x] Implementing Proxy(forwarder) Side Allowed Relay Hostname Pattern Indication
- [x] Creating Testing Environment for Distributed Snowflake Server
- [x] Implementing Broker Side Allowed Relay Hostname Pattern Indication Rejection for Proxy
- [x] Implementing Broker Side Allowed Relay Hostname Pattern Indication Rejection for Proxy - Better Error Message
- [x] Make sure legacy client will still work
- [x] Make sure legacy client config on new client will still work
- [x] Make sure legacy proxy will still work(to a limited degree)
- [x] Add Metrics for Proxy Relay URL Extension Support Status.
- [ ] Implementing Broker Side Allowed Relay Hostname Pattern Indication Rejection for Server
- [x] Implementing Web Proxy(forwarder) Custom Relay URL Support
- [x] Implementing Web Proxy(forwarder) Custom Relay URL Hostname Pattern Matching Guard
- [x] Implementing Web Proxy(forwarder) Side Allowed Relay Hostname Pattern Indication
- [ ] Implementing Web Proxy(forwarder) Relay URL Hostname Pattern UI
- [ ] User Document for Distributed Snowflake Server - Proxy Operators
- [ ] User Document for Distributed Snowflake Server - Client Users
- [x] Setup a Second Snowflake Bridge
### WIP Branch ###
Distributed Snowflake Testing Environment: https://github.com/xiaokangwang/snowflake-mu-docker
Distributed Snowflake: https://gitlab.torproject.org/shelikhoo/snowflake/-/commits/dev-mubrokershelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40128Give standalone snowflakes guidance on how best to set up their nat2023-03-31T16:56:08ZRoger DingledineGive standalone snowflakes guidance on how best to set up their natAccording to our current broker stats (https://snowflake-broker.torproject.net/debug), we have
```
current snowflakes available: 3021
standalone proxies: 2589
browser proxies: 5
webext proxies: 250
unknown proxies: 177
NAT Types avai...According to our current broker stats (https://snowflake-broker.torproject.net/debug), we have
```
current snowflakes available: 3021
standalone proxies: 2589
browser proxies: 5
webext proxies: 250
unknown proxies: 177
NAT Types available:
restricted: 2512
unrestricted: 386
unknown: 123
```
i.e. most of the snowflakes that we're giving out seem to be standalone ones as opposed to browser extension ones, and also most of the ones we have available to us are behind restricted nat.
It seems to me that the standalone ones are probably in a better position to be behind the good kind of nat (or no nat at all). But does our docker image impose the bad kind of nat on them by default? How come so many standalone proxies are behind restricted nat?
More generally: is there useful guidance we can give people, on setting themselves up with the right kind of nat, presuming they're on a VPS or otherwise on a 'real' internet connection?shelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40127Bump webrtc and dtls library versions for Snowflake2022-04-14T13:51:52ZCecylia BocovichBump webrtc and dtls library versions for SnowflakeIt's been awhile since we bumped the version of webrtc and dtls.
The most recent release of https://github.com/pion/webrtc/ is `v3.1.28`. It includes the most recent version of https://github.com/pion/dtls/, which is `v2.1.3`. This incl...It's been awhile since we bumped the version of webrtc and dtls.
The most recent release of https://github.com/pion/webrtc/ is `v3.1.28`. It includes the most recent version of https://github.com/pion/dtls/, which is `v2.1.3`. This includes the fingerprinting fixes we made to circumvent blocking in Russia (see #40014).
We should do a quick look at the other changes made and test it out to make sure everything works as we expect.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40126Visualize the Snowflake Network2023-03-31T19:00:35ZsereneVisualize the Snowflake NetworkTo further upgrade the main page, we could include a live visualization showing the strength of the Snowflake Network. (in a way which is scrubbed / anonymized of course, which the underlying metrics I believe always are.)
- It can show...To further upgrade the main page, we could include a live visualization showing the strength of the Snowflake Network. (in a way which is scrubbed / anonymized of course, which the underlying metrics I believe always are.)
- It can show how many people are currently helping, how many people are being helped.
- It would further assist in immediately making clear to new visitors what exactly Snowflake is/does, and how they can immediately be involved... whether as a volunteer proxy, as a user, as a dev, as a funder...
- It would cool.
I've not implemented this yet, but it's on my list. It would be an excellent addition to the landing page. See: #40125
I will update this ticket with a screenshot or demo soon.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40124Move tor-specific event code outside of library2022-07-28T14:38:06ZCecylia BocovichMove tor-specific event code outside of libraryThere was a slight regression of our library goals (https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/31) in the fix for https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transpor...There was a slight regression of our library goals (https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/31) in the fix for https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40076.
The goal of separating out the client and server libraries were to:
- implement v2 of the pluggable transports Go API
- allow non-Tor programs to run Snowflake
The cleanest way to do this is the separate the Tor-specific code into the main program that calls the library. Right now there are calls to the tor pt v1 specification in `pt_event_logger.go` inside the client library that will attempt to send log messages to a tor process if used. I'd like to just move this file out of the library. Should be a simple fix.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40123Multicast DNS noise2023-02-07T04:10:48ZpseudonymisaTorMulticast DNS noiseFrom Firewall logs, I see ❄️ Snowflake client try to create exactly one connection per every second to the 224.0.0.251:5353 well-known multicast address for multicast Domain Name System (mDNS) from any available interface as source.
Whi...From Firewall logs, I see ❄️ Snowflake client try to create exactly one connection per every second to the 224.0.0.251:5353 well-known multicast address for multicast Domain Name System (mDNS) from any available interface as source.
While searching for the reason, I just found: [Detecting Snowflake](https://www.hackerfactor.com/blog/index.php?/archives/944-Tor-0day-Snowflake.html) TLDR:
> Regular WebRTC clients do not do hostname lookups for remote STUN servers on the local network. If you see any DNS lookups for snowflake's STUN servers on the local network (stun.epygi.com.internal.lan, stun.voipgate.com.internal.lan, etc.) then you've found a Tor snowflake client.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40122Set up a second snowflake bridge site2023-01-08T18:03:25ZDavid Fifielddcf@torproject.orgSet up a second snowflake bridge siteAt #28651, the plan is to run more than one snowflake bridge site. Each bridge site will have one instance of snowflake-server, speaking to a group of tor instances that all have the same identity keys. The identity keys in different bri...At #28651, the plan is to run more than one snowflake bridge site. Each bridge site will have one instance of snowflake-server, speaking to a group of tor instances that all have the same identity keys. The identity keys in different bridge sites will be different. Until now, we have had only one bridge site, and have kept the tor identity keys the same through server migrations (#40091, #40095, #40110, #40111). This issue is to set up a second bridge site, with its own, independent identity keys.
- [x] get access to the server hardware
- [x] decide on a bridge nickname
- [x] install bridge software ([installation guide](https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Survival-Guides/Snowflake-Bridge-Installation-Guide))
- [x] set up user accounts https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40091#note_2768855
- [x] point a domain name at the server https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/28651#note_2787394
- [x] disable plain SSH access
/cc @shelikhooDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40121add prometheus support to snowflake proxy2024-02-14T16:48:40Zcypherpunksadd prometheus support to snowflake proxyfrom today's relay meetup:
For better maintainability and service monitoring please add a prometheus exporter to snowflake proxy with at least the following data:
- bandwidth
- memory usage
- uptime
- sockets/connections
- version
Th...from today's relay meetup:
For better maintainability and service monitoring please add a prometheus exporter to snowflake proxy with at least the following data:
- bandwidth
- memory usage
- uptime
- sockets/connections
- version
This will allow us to detect when the service crashed and got restarted or uses significantly less/more bw/memory/sockets.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40120Certificate failure in Russia2022-06-20T14:54:34ZcypherpunksCertificate failure in Russia(I wonder whether it is correct place to report)
Latest TBB with snowflake currently fails to connect in Russia. It bootstraps only to 10% and then repeatedly logs:
offer created
broker failure x509: certificate has expired or is not ye...(I wonder whether it is correct place to report)
Latest TBB with snowflake currently fails to connect in Russia. It bootstraps only to 10% and then repeatedly logs:
offer created
broker failure x509: certificate has expired or is not yet valid:
(yes, with colon at the end. Looks like there should be some details)
I can access broker both directly and over Fastly using my normal browser.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40119let user bind interface honor OutboundBindAddressPT2022-04-11T21:45:27ZpseudonymisaTorlet user bind interface honor OutboundBindAddressPTCurrently, snowflake client does ignore torrc `OutboundBindAddressPT` option.
```
# [[OutboundBindAddressPT]] **OutboundBindAddressPT** __IP__::
# Request that pluggable transports makes all outbound connections
# originate fro...Currently, snowflake client does ignore torrc `OutboundBindAddressPT` option.
```
# [[OutboundBindAddressPT]] **OutboundBindAddressPT** __IP__::
# Request that pluggable transports makes all outbound connections
# originate from the IP address specified. Because outgoing connections
# are handled by the pluggable transport itself, it is not possible for
# Tor to enforce whether the pluggable transport honors this option. This
# option overrides **OutboundBindAddress** for the same IP version. This
# option may be used twice, once with an IPv4 address and once with an
# IPv6 address. IPv6 addresses should be wrapped in square brackets. This
# setting will be ignored for connections to the loopback addresses
# (127.0.0.0/8 and ::1).
```https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40118Fix misleading proxy usage statistics message on launch2022-05-31T20:36:21Zmeskiomeskio@torproject.orgFix misleading proxy usage statistics message on launchAs soon as you launch the proxy it displays:
```
2022/03/23 09:27:18 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
```
I guess it will be better to don't display that until some time has actually passed. Or ...As soon as you launch the proxy it displays:
```
2022/03/23 09:27:18 In the last 1h0m0s, there were 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.
```
I guess it will be better to don't display that until some time has actually passed. Or at least don't say `In the last 1h`, because it hasn't being running 1hour.itchyonionitchyonionhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40117display the proxy NAT type in the logs2022-11-16T18:19:45Zmeskiomeskio@torproject.orgdisplay the proxy NAT type in the logsThe proxy NAT type is only being written to the logs if the `-verbose` flag is set. Will be nice to display it anyway.The proxy NAT type is only being written to the logs if the `-verbose` flag is set. Will be nice to display it anyway.itchyonionitchyonionhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40116Soften Tor log output for non critical events2022-04-12T15:57:06ZCecylia BocovichSoften Tor log output for non critical eventsWe had a forum post from a user who interpreted one of the snowflake connection events as a critical failure of snowflake: https://forum.torproject.net/t/snowflake-does-not-work-anymore/2650/2
While a failure to connect to the broker ca...We had a forum post from a user who interpreted one of the snowflake connection events as a critical failure of snowflake: https://forum.torproject.net/t/snowflake-does-not-work-anymore/2650/2
While a failure to connect to the broker can be critical, a failure to open a data channel with a snowflake is not unusual and snowflake can easily recover from it. Let's make a small change of the log message from "connection failed" to "trying a new proxy: [error message]" or something like thatitchyonionitchyonionhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40115Scrub pt.Log calls like other logs2022-11-07T16:25:28ZDavid Fifielddcf@torproject.orgScrub pt.Log calls like other logs!67 added [`ptEventLogger`](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/bd636a1374efb514bbc40acbd1dcaf0ecec26916/client/lib/pt_event_logger.go) which sends messages to the managing process usin...!67 added [`ptEventLogger`](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/bd636a1374efb514bbc40acbd1dcaf0ecec26916/client/lib/pt_event_logger.go) which sends messages to the managing process using `pt.Log`. But these logs are not scrubbed of IP addresses the way all other logs are scrubbed (as in
#21304).
I saw this in the Tor Logs in Tor Browser:
```
3/17/22, 02:24:50.145 [NOTICE] Managed proxy "./TorBrowser/Tor/PluggableTransports/snowflake-client": offer created
3/17/22, 02:24:50.146 [NOTICE] Managed proxy "./TorBrowser/Tor/PluggableTransports/snowflake-client": broker failure dial tcp: lookup cdn.sstatic.net on 192.168.0.1:53: dial udp 192.168.0.1:53: connect: network is unreachable
```itchyonionitchyonionhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40114No connections (there are 0 connections) after running for for a few days2022-03-25T11:41:19ZGalvaniObstNo connections (there are 0 connections) after running for for a few daysHello,
I've seen multiple times the issue that my Snowflake proxy says "In the last 1h0m0s, there are 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.".
I'm running Snowflake in Docker and it's serving between 1-10 connections each hour. W...Hello,
I've seen multiple times the issue that my Snowflake proxy says "In the last 1h0m0s, there are 0 connections. Traffic Relayed ↑ 0 B, ↓ 0 B.".
I'm running Snowflake in Docker and it's serving between 1-10 connections each hour. What I noticed is that after running for multiple days (3-5) it doesn't serve any connections.
By restarting the docker container the issue is solved. In the first hour after the restart I see connections but after a few days the issue reoccurs.
Is this expected behaviour that the proxy has periods where it doesn't server connections für >24 hours?
Is the restart required or would it server connections if I wait longer (>24h)?https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40113Surveying Snowflake CPU/RAM utilization2023-06-21T09:15:10ZGeorgeSurveying Snowflake CPU/RAM utilizationAs per issue raised at 20220305 Relay Operators meetup, we discussed possible high CPU usage with Snowflake with browser addon. Relates to "High CPU load on idle proxies" #40112
We should get contributions of users detailing:
snowflake...As per issue raised at 20220305 Relay Operators meetup, we discussed possible high CPU usage with Snowflake with browser addon. Relates to "High CPU load on idle proxies" #40112
We should get contributions of users detailing:
snowflake_version,browser,browser_version,installed_addons,CPU,CPU_snowflake_usage,RAM,RAM_snowflake_usage,operating_system
We should likely include average Snowflake usage and maybe consider basic hardware specs on device. Contributions should no have other applications running to better control the results.
Any enhancements on this survey welcome.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40112High CPU load on idle proxies2024-03-05T18:23:24ZPeter GerberHigh CPU load on idle proxiesI've been running Snowflake proxies for some time now and I'm seeing rather high CPU load even when proxies are idle.
I'm seeing CPU usage similar to this on all my proxies:
```
systemctl status tor-snowflake
● tor-snowflake.service - ...I've been running Snowflake proxies for some time now and I'm seeing rather high CPU load even when proxies are idle.
I'm seeing CPU usage similar to this on all my proxies:
```
systemctl status tor-snowflake
● tor-snowflake.service - Tor Snowflake bridge
Loaded: loaded (/etc/systemd/system/tor-snowflake.service; enabled; vendor preset: enabled)
Active: active (running) since Sat 2022-03-12 11:18:37 CET; 1 day 5h ago
Main PID: 3242240 (tor-snowflake)
IP: 1.5G in, 1.5G out
Tasks: 9 (limit: 19046)
Memory: 55.6M
CPU: 7h 59min 35.486s
CGroup: /system.slice/tor-snowflake.service
└─3242240 /usr/local/bin/tor-snowflake
```
This corresponds to about a **~28% CPU load at an average speed of 15 kiB/s**.
I looked at the CPU load over the timespan of a few days. I see a CPU load of around 30% even with no traffic whatsoever. Thus, I suspect there are CPU cycles wasted somewhere.
OS: Debian 11 "bullseye"
Version: 19e9e384154adc6251579dc6843f11f53cbd0146https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40111Move bridge to a permanent faster server2022-10-19T20:40:14ZDavid Fifielddcf@torproject.orgMove bridge to a permanent faster serverBackground: [\[tor-project\] More resources required for Snowflake bridge](https://forum.torproject.net/t/tor-project-more-resources-required-for-snowflake-bridge/2353)
I expect to be able to move the snowflake bridge to a more permanen...Background: [\[tor-project\] More resources required for Snowflake bridge](https://forum.torproject.net/t/tor-project-more-resources-required-for-snowflake-bridge/2353)
I expect to be able to move the snowflake bridge to a more permanent home on a faster server after 2022-03-21.
#40110 is to use a *different* faster server in the meantime, until the permanent one is prepared.
- [x] get access to new server hardware
- [x] install new bridge ([installation guide](https://gitlab.torproject.org/tpo/anti-censorship/team/-/wikis/Survival-Guides/Snowflake-Bridge-Installation-Guide))
- [x] copy user accounts https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40091#note_2768855
- [x] copy identity and onion keys from existing bridge
- [x] double check onion keys
```
# md5sum /var/lib/tor-instances/*/keys/secret_onion_key{,_ntor}
f57a05262f65beea15ec05bbeefe404c /var/lib/tor-instances/snowflake1/keys/secret_onion_key
a16c5403d18509c79fa7b863eb66892a /var/lib/tor-instances/snowflake1/keys/secret_onion_key_ntor
```
- [x] copy HTTPS TLS keys and certificates from existing bridge
- [x] test tor bootstrap on new bridge using local broker and proxy, and /etc/hosts domain name record https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40095#note_2773365
- [x] test rebooting the server to make sure everything comes back up
- [x] start the tor@snowflake* services
- [x] make DNS for snowflake.torproject.net point to the new bridge tpo/tpa/team#40716
- [x] monitor for a day, be ready to switch DNS back if connections fail
- [x] after a week or so, shut down temporary bridge
Cc @linusDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.org