Snowflake issueshttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues2024-03-21T20:15:25Zhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40352Use unreliable and unordered WebRTC data channels2024-03-21T20:15:25ZDavid Fifielddcf@torproject.orgUse unreliable and unordered WebRTC data channels@shelikhoo:
Actually here are some observation from me related to https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40243:
Snowflake is currently using network resource in a so suboptimal way I ...@shelikhoo:
Actually here are some observation from me related to https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40243:
Snowflake is currently using network resource in a so suboptimal way I think it would make sense to also consider make protocol level change on how kcp is interacting with webrtc before considering to add forward error correction. This would be in the form of enabling unreliable mode of webrtc and make necessary change to get it to work.
Right now, kcp packets are sent in webrtc data channel in a reliable way that deliver all packets in order and retransmit any lost message repeatedly. However, kcp also retransmit its packet itself, which as a result, queue all those retransmitted packets somewhere, like in webrtc's buffer.
This means lost packets are required to be retransmitted more than once in different protocol, and retransmit & timeout get compounded. More retransmit result in more lost packets and more retransmission, which eventually lead to [connection melt down](https://openvpn.net/faq/what-is-tcp-meltdown/) <- please read.
back pressure like the one introduced in https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/merge_requests/144 only move the problem, and block kcp's send in unexpected way, as kcp don't expect send to block as it is usually over udp.
See also: https://lists.torproject.org/pipermail/anti-censorship-team/2023-March/000286.html
(@dcf split this issue off from #40251 to separate the analysis of speed in China from the proposed remedy.)shelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40251Analysis of speed deficiency of Snowflake in China, 2023 Q12024-03-21T20:45:13ZshelikhooAnalysis of speed deficiency of Snowflake in China, 2023 Q1We are currently observing an increase of snowflake bootstrap failure. This ticket document our investigation of this incident.
As we can observe from the vantage point test [result](https://gitlab.torproject.org/tpo/anti-censorship/con...We are currently observing an increase of snowflake bootstrap failure. This ticket document our investigation of this incident.
As we can observe from the vantage point test [result](https://gitlab.torproject.org/tpo/anti-censorship/connectivity-measurement/bridgestatus/-/blob/39c4d2a143c2ce43ffb1cbf39bf18f26d7ba49c7/recentResult_cn), the bootstrap percentage is often more than 10 and less than 100 as a result of poor connection speed.
In order to measure the packet loss rate at the vantage points a few [scripts](https://gist.github.com/xiaokangwang/14ac48ef9fc2ce8dd04f92ed9c0928de) are used to calculate the packet loss rate from packet capture file, here is the result:
```
snowflake-probe-0-eth0.pcap:TOTAL 3027, RECV 2702, LOSS RATE .107
snowflake-probe-1-eth0.pcap:TOTAL 3406, RECV 3169, LOSS RATE .069
snowflake-probe-2-eth0.pcap:TOTAL 2896, RECV 2294, LOSS RATE .207
snowflake-probe-3-eth0.pcap:TOTAL 2883, RECV 2652, LOSS RATE .080
snowflake-probe-4-eth0.pcap:TOTAL 2696, RECV 2514, LOSS RATE .067
snowflake-probe-5-eth0.pcap:TOTAL 847, RECV 669, LOSS RATE .210
snowflake-probe-6-eth0.pcap:TOTAL 1855, RECV 1692, LOSS RATE .087
snowflake-probe-7-eth0.pcap:TOTAL 76, RECV 284, LOSS RATE -2.736 (invalid, more than one dtls connection)
snowflake-probe-8-eth0.pcap:TOTAL 1577, RECV 1255, LOSS RATE .204
snowflake-probe-9-eth0.pcap:TOTAL 1449, RECV 1166, LOSS RATE .195
```
As we can see snowflake's bootstrap percentage is regularly impacted by packet loss rate. We can either make snowflake more resistant to packet loss or improve matching process to reduce packet loss.shelikhooshelikhoo