Anti-censorship issueshttps://gitlab.torproject.org/groups/tpo/anti-censorship/-/issues2024-02-13T21:15:19Zhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake-webext/-/issues/82WS.makeWebsocket ignores params (i.e. `client_ip`), losing country statistics2024-02-13T21:15:19ZDavid Fifielddcf@torproject.orgWS.makeWebsocket ignores params (i.e. `client_ip`), losing country statisticsI am investigating a sudden partial loss of [`client_ip` statistics](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/18628) that occurred after 2022-06-27,
when the fraction of connections that h...I am investigating a sudden partial loss of [`client_ip` statistics](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/18628) that occurred after 2022-06-27,
when the fraction of connections that had `client_ip` set dropped from 99% to 94%.
I have tracked it down to commit 15768f50c0ddd68d3ffb815cd532ddbd3d85fd41, part of !29,
which was released in [snowflake-webextension-0.6.0](https://archive.org/details/snowflake-webextension-0.6.0)
on the day in question, 2022-06-27.
The bug is in [`WS.makeWebsocket`](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake-webext/-/blob/995a42710044ae1df0fb839b12906049549c2074/websocket.js#L13-29).
Notice how `params` is used to create `parsedURL`—but then `parsedURL` is thrown away
and the `WebSocket` constructor is called on the unmodified `url` argument:
```js
static makeWebsocket(url, params) {
let parsedURL = new URL(url);
let urlpa = new URLSearchParams(params);
urlpa.forEach(function (value, key) {
parsedURL.searchParams.set(key, value);
});
let ws = new WebSocket(url);
ws.binaryType = 'arraybuffer';
return ws;
}
```
The effect of this bug is a partial loss of client IP geolocation statistics at the bridge.
Connections will be attributed to `??` rather than the proper country code.
snowflake-server [logs once a day](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/commit/d9e8f8f6479a8b4abe31eb5bfb9023de1bbca8af)
the number of connections that had `client_ip` set:
```
$ grep 'connections had client_ip$' /var/log/snowflake-server/snowflake-server.log
2022/06/25 17:53:16 in the past 86400 s, 194793/195854 connections had client_ip
2022/06/26 17:53:16 in the past 86400 s, 194526/195608 connections had client_ip
2022/06/27 17:53:16 in the past 86400 s, 185234/187519 connections had client_ip
2022/06/28 17:53:16 in the past 86400 s, 172629/183925 connections had client_ip
2022/06/29 17:53:16 in the past 86400 s, 169512/180011 connections had client_ip
2022/06/30 17:53:16 in the past 86400 s, 162445/172427 connections had client_ip
```
Converting to percentages, we see that after 2022-06-27 the proportion dropped
from 99+% to about 94%:
```
$ grep 'connections had client_ip$' /var/log/snowflake-server/snowflake-server.log | perl -ane '$F[7] =~ m#^(.*)/(.*)$#; printf("%s %s %8d/%-8d %6.2f%%\n", $F[0], $F[1], $1, $2, 100 * ($2 == 0 ? 0.0 : $1/$2));'
2022/06/25 17:53:16 194793/195854 99.46%
2022/06/26 17:53:16 194526/195608 99.45%
2022/06/27 17:53:16 185234/187519 98.78%
2022/06/28 17:53:16 172629/183925 93.86%
2022/06/29 17:53:16 169512/180011 94.17%
2022/06/30 17:53:16 162445/172427 94.21%
```
<details>
<summary>Full percentage log</summary>
```
2022/04/07 17:53:16 4/4 100.00%
2022/04/08 17:53:16 0/0 0.00%
2022/04/09 17:53:16 0/0 0.00%
2022/04/10 17:53:16 0/0 0.00%
2022/04/11 17:53:16 14755/14846 99.39%
2022/04/12 17:53:16 155044/156119 99.31%
2022/04/13 17:53:16 151189/152289 99.28%
2022/04/14 17:53:16 151366/152522 99.24%
2022/04/15 17:53:16 159661/160890 99.24%
2022/04/16 17:53:16 165119/166343 99.26%
2022/04/17 17:53:16 170390/171655 99.26%
2022/04/18 17:53:16 158072/159215 99.28%
2022/04/19 17:53:16 155255/156348 99.30%
2022/04/20 17:53:16 153641/154624 99.36%
2022/04/21 17:53:16 154686/155618 99.40%
2022/04/22 17:53:16 166803/167743 99.44%
2022/04/23 17:53:16 170969/172005 99.40%
2022/04/24 17:53:16 163772/164752 99.41%
2022/04/25 17:53:16 164409/165359 99.43%
2022/04/26 17:53:16 161246/162234 99.39%
2022/04/27 17:53:16 161217/162123 99.44%
2022/04/28 17:53:16 161105/162029 99.43%
2022/04/29 17:53:16 167518/168391 99.48%
2022/04/30 17:53:16 174056/174942 99.49%
2022/05/01 17:53:16 172758/173729 99.44%
2022/05/02 17:53:16 171274/172289 99.41%
2022/05/03 17:53:16 174248/175357 99.37%
2022/05/04 17:53:16 177377/178406 99.42%
2022/05/05 17:53:16 174350/175339 99.44%
2022/05/06 17:53:16 179096/180160 99.41%
2022/05/07 17:53:16 181913/182982 99.42%
2022/05/08 17:53:16 175225/176240 99.42%
2022/05/09 17:53:16 190505/191629 99.41%
2022/05/10 17:53:16 189615/190739 99.41%
2022/05/11 17:53:16 180694/181761 99.41%
2022/05/12 17:53:16 171556/172619 99.38%
2022/05/13 17:53:16 172153/173163 99.42%
2022/05/14 17:53:16 180281/181201 99.49%
2022/05/15 17:53:16 176023/177015 99.44%
2022/05/16 17:53:16 172430/173301 99.50%
2022/05/17 17:53:16 171169/172126 99.44%
2022/05/18 17:53:16 175336/176319 99.44%
2022/05/19 17:53:16 177217/178199 99.45%
2022/05/20 17:53:16 176879/177677 99.55%
2022/05/21 17:53:16 187563/188484 99.51%
2022/05/22 17:53:16 190738/191674 99.51%
2022/05/23 17:53:16 180016/180821 99.55%
2022/05/24 17:53:16 184586/185499 99.51%
2022/05/25 17:53:16 182584/183455 99.53%
2022/05/26 17:53:16 185874/186754 99.53%
2022/05/27 17:53:16 185319/186194 99.53%
2022/05/28 17:53:16 190375/191119 99.61%
2022/05/29 17:53:16 186398/187169 99.59%
2022/05/30 17:53:16 184392/185282 99.52%
2022/05/31 17:53:16 181668/182598 99.49%
2022/06/01 17:53:16 182936/183759 99.55%
2022/06/02 17:53:16 189689/190490 99.58%
2022/06/03 17:53:16 193557/194353 99.59%
2022/06/04 17:53:16 194960/195770 99.59%
2022/06/05 17:53:16 194091/194822 99.62%
2022/06/06 17:53:16 191104/191859 99.61%
2022/06/07 17:53:16 191891/192723 99.57%
2022/06/08 17:53:16 194276/194989 99.63%
2022/06/09 17:53:16 196756/197772 99.49%
2022/06/10 17:53:16 188985/189979 99.48%
2022/06/11 17:53:16 189916/190888 99.49%
2022/06/12 17:53:16 185694/186730 99.45%
2022/06/13 17:53:16 206684/207989 99.37%
2022/06/14 17:53:16 207371/208717 99.36%
2022/06/15 17:53:16 199041/200315 99.36%
2022/06/16 17:53:16 189537/190608 99.44%
2022/06/17 17:53:16 185730/186857 99.40%
2022/06/18 17:53:16 198655/199636 99.51%
2022/06/19 17:53:16 192534/193580 99.46%
2022/06/20 17:53:16 196991/198095 99.44%
2022/06/21 17:53:16 184941/185865 99.50%
2022/06/22 17:53:16 183693/184676 99.47%
2022/06/23 17:53:16 181582/182624 99.43%
2022/06/24 17:53:16 184876/185957 99.42%
2022/06/25 17:53:16 194793/195854 99.46%
2022/06/26 17:53:16 194526/195608 99.45%
2022/06/27 17:53:16 185234/187519 98.78%
2022/06/28 17:53:16 172629/183925 93.86%
2022/06/29 17:53:16 169512/180011 94.17%
2022/06/30 17:53:16 162445/172427 94.21%
2022/07/01 17:53:16 166284/176521 94.20%
2022/07/02 17:53:16 169610/179069 94.72%
2022/07/03 17:53:16 176635/186127 94.90%
2022/07/04 17:53:16 168869/179203 94.23%
2022/07/05 17:53:16 163400/173719 94.06%
2022/07/06 17:53:16 171810/182838 93.97%
2022/07/07 17:53:16 170560/180939 94.26%
2022/07/08 17:53:16 169999/180116 94.38%
2022/07/09 17:53:16 170241/179759 94.71%
2022/07/10 17:53:16 167606/176432 95.00%
2022/07/11 17:53:16 168153/178160 94.38%
2022/07/12 17:53:16 170696/180506 94.57%
2022/07/13 17:53:16 169343/179592 94.29%
2022/07/14 17:53:16 171230/181636 94.27%
2022/07/15 17:53:16 184074/195135 94.33%
2022/07/16 17:53:16 192010/202251 94.94%
2022/07/17 17:53:16 196606/206267 95.32%
2022/07/18 17:53:16 200790/212053 94.69%
2022/07/19 17:53:16 197761/209221 94.52%
2022/07/20 17:53:16 196473/207642 94.62%
2022/07/21 17:53:16 204488/215694 94.80%
2022/07/22 17:53:16 220073/232393 94.70%
2022/07/23 17:53:16 219360/230541 95.15%
2022/07/24 17:53:16 213575/224549 95.11%
2022/07/25 17:53:16 205768/219132 93.90%
2022/07/26 17:53:16 211755/227503 93.08%
2022/07/27 17:53:16 215612/229109 94.11%
2022/07/28 17:53:16 224365/238228 94.18%
2022/07/29 17:53:16 222637/237054 93.92%
2022/07/30 17:53:16 216264/228013 94.85%
2022/07/31 17:53:16 220990/233336 94.71%
2022/08/01 17:53:16 221660/237294 93.41%
2022/08/02 17:53:16 216055/234745 92.04%
2022/08/03 17:53:16 222963/244030 91.37%
2022/08/04 17:53:16 224417/245564 91.39%
2022/08/05 17:53:16 228494/247388 92.36%
2022/08/06 17:53:16 223448/238546 93.67%
2022/08/07 17:53:16 233452/248800 93.83%
2022/08/08 17:53:16 228015/244361 93.31%
2022/08/09 17:53:16 226362/242047 93.52%
2022/08/10 17:53:16 224323/240293 93.35%
2022/08/11 17:53:16 223786/239403 93.48%
2022/08/12 17:53:16 229123/245179 93.45%
2022/08/13 17:53:16 228316/242586 94.12%
2022/08/14 17:53:16 226738/241042 94.07%
2022/08/15 17:53:16 228205/243680 93.65%
2022/08/16 17:53:16 229760/246054 93.38%
2022/08/17 17:53:16 227155/243094 93.44%
2022/08/18 17:53:16 227517/243172 93.56%
2022/08/19 17:53:16 227941/243997 93.42%
2022/08/20 17:53:16 228990/243900 93.89%
2022/08/21 17:53:16 230668/245426 93.99%
2022/08/22 17:53:16 224561/240036 93.55%
2022/08/23 17:53:16 219832/235655 93.29%
2022/08/24 17:53:16 222124/238295 93.21%
2022/08/25 17:53:16 223891/240074 93.26%
2022/08/26 17:53:16 223819/240527 93.05%
2022/08/27 17:53:16 228663/243568 93.88%
2022/08/28 17:53:16 225569/239817 94.06%
2022/08/29 17:53:16 224327/240134 93.42%
2022/08/30 17:53:16 228673/245654 93.09%
2022/08/31 17:53:16 226735/243293 93.19%
2022/09/01 17:53:16 225639/241630 93.38%
2022/09/02 17:53:16 236206/252896 93.40%
2022/09/03 17:53:16 233258/248503 93.87%
2022/09/04 17:53:16 232155/247386 93.84%
2022/09/05 17:53:16 234814/251423 93.39%
2022/09/06 17:53:16 232808/249994 93.13%
2022/09/07 17:53:16 240543/257957 93.25%
2022/09/08 17:53:16 231430/248035 93.31%
2022/09/09 17:53:16 242153/259413 93.35%
2022/09/10 17:53:16 243891/260512 93.62%
2022/09/11 17:53:16 248028/265232 93.51%
2022/09/12 17:53:16 242159/260327 93.02%
2022/09/13 17:53:16 239257/257526 92.91%
2022/09/14 17:53:16 239152/257146 93.00%
2022/09/15 17:53:16 231770/249919 92.74%
2022/09/16 17:53:16 234916/254471 92.32%
2022/09/17 17:53:16 226918/245014 92.61%
2022/09/18 17:53:16 227741/245597 92.73%
2022/09/19 17:53:16 216576/233794 92.64%
2022/09/20 17:53:16 216902/234266 92.59%
2022/09/21 17:53:16 221773/239192 92.72%
2022/09/25 21:34:29 1121561/1260527 88.98%
2022/09/28 02:02:49 1381811/1532420 90.17%
2022/10/03 10:07:21 3700252/3968879 93.23%
2022/10/04 10:07:21 2701111/3187521 84.74%
2022/10/06 01:10:13 878980/1149771 76.45%
2022/10/07 19:51:05 687493/936395 73.42%
2022/10/08 19:51:05 557700/769207 72.50%
2022/10/09 19:51:05 652399/856848 76.14%
2022/10/10 19:51:05 645157/862235 74.82%
2022/10/11 19:51:05 665194/882964 75.34%
2022/10/12 19:51:05 452181/651005 69.46%
2022/10/13 19:51:05 615747/844214 72.94%
2022/10/14 19:51:05 543988/760253 71.55%
2022/10/15 19:51:05 419633/597319 70.25%
2022/10/16 19:51:05 532650/731640 72.80%
2022/10/17 19:51:05 597205/786802 75.90%
2022/10/18 19:51:05 545984/715618 76.30%
2022/10/20 07:31:25 574485/676737 84.89%
2022/10/21 07:31:25 554463/675893 82.03%
2022/10/22 07:31:25 518909/671584 77.27%
2022/10/23 07:31:25 541626/691770 78.30%
2022/10/24 07:31:25 538160/689136 78.09%
2022/10/25 07:31:25 510459/656238 77.79%
2022/10/27 07:00:29 518697/614072 84.47%
2022/10/28 07:00:29 486259/608324 79.93%
2022/10/29 07:00:29 491426/630300 77.97%
2022/10/30 07:00:29 472230/597290 79.06%
2022/10/31 07:00:29 491910/624836 78.73%
2022/11/01 07:00:29 475916/639245 74.45%
2022/11/02 07:00:29 481206/646550 74.43%
2022/11/03 07:00:29 599719/776774 77.21%
2022/11/04 07:00:29 737956/925233 79.76%
2022/11/05 07:00:29 873922/1093662 79.91%
2022/11/06 07:00:29 879774/1077541 81.65%
2022/11/07 07:00:29 905093/1113103 81.31%
2022/11/08 07:00:29 935191/1153421 81.08%
2022/11/09 07:00:29 1074279/1308056 82.13%
2022/11/10 07:00:29 966623/1181925 81.78%
2022/11/11 07:00:29 1031706/1266995 81.43%
2022/11/12 07:00:29 1069843/1285152 83.25%
2022/11/13 07:00:29 999539/1248679 80.05%
2022/11/14 07:00:29 988052/1222193 80.84%
2022/11/15 07:00:29 951821/1191297 79.90%
2022/11/16 07:00:29 1110988/1329583 83.56%
2022/11/17 07:00:29 1117195/1324894 84.32%
2022/11/18 07:00:29 1173611/1379627 85.07%
2022/11/19 07:00:29 1205208/1414808 85.19%
2022/11/20 07:00:29 1189146/1398402 85.04%
2022/11/21 07:00:29 1186982/1495221 79.39%
2022/11/22 07:00:29 1081288/1385520 78.04%
2022/11/23 07:00:29 1244621/1587082 78.42%
2022/11/24 07:00:29 1341024/1674905 80.07%
2022/11/25 07:00:29 1421500/1757062 80.90%
2022/11/26 07:00:29 1560783/1904455 81.95%
2022/11/27 07:00:29 1465706/1671865 87.67%
2022/11/28 07:00:29 1429704/1643762 86.98%
2022/11/29 07:00:29 1457300/1700892 85.68%
2022/11/30 07:00:29 1487931/1720901 86.46%
2022/12/01 07:00:29 1506246/1740213 86.56%
2022/12/02 07:00:29 1460951/1683721 86.77%
2022/12/03 07:00:29 1485571/1703312 87.22%
2022/12/04 07:00:29 1438832/1627932 88.38%
2022/12/05 07:00:29 1323146/1522810 86.89%
2022/12/06 07:00:29 1382024/1609148 85.89%
2022/12/07 07:00:29 1442358/1645869 87.64%
2022/12/08 07:00:29 1736377/1930441 89.95%
2022/12/09 07:00:29 1870385/2047066 91.37%
2022/12/10 07:00:29 1807694/1985861 91.03%
2022/12/11 07:00:29 1798714/1958147 91.86%
2022/12/12 07:00:29 1788181/1951137 91.65%
2022/12/13 07:00:29 1831817/2011966 91.05%
2022/12/14 07:00:29 1963352/2149952 91.32%
2022/12/15 07:00:29 1957450/2146125 91.21%
2022/12/16 07:00:29 2008079/2193701 91.54%
2022/12/17 07:00:29 2405453/2583256 93.12%
2022/12/18 07:00:29 2652740/2823784 93.94%
2022/12/19 07:00:29 2464515/2634705 93.54%
2022/12/20 07:00:29 2589562/2776041 93.28%
2022/12/21 07:00:29 2556656/2727805 93.73%
2022/12/22 07:00:29 2621775/2790903 93.94%
2022/12/23 07:00:29 2612666/2786059 93.78%
2022/12/24 07:00:29 2702727/2863852 94.37%
2022/12/25 07:00:29 2699413/2825448 95.54%
2022/12/26 07:00:29 2695487/2811637 95.87%
2022/12/27 07:00:29 2716205/2840448 95.63%
2022/12/28 23:36:14 2914171/3045553 95.69%
2022/12/29 23:36:14 2898755/3043905 95.23%
2022/12/30 23:36:14 2929119/3093004 94.70%
2022/12/31 23:36:14 2963652/3115697 95.12%
2023/01/01 23:36:14 3019037/3183507 94.83%
2023/01/02 23:36:14 2980591/3177055 93.82%
2023/01/03 23:36:14 3001436/3207975 93.56%
2023/01/05 13:39:26 3078681/3250828 94.70%
2023/01/06 13:39:26 3012651/3191445 94.40%
2023/01/07 13:39:26 3039471/3206161 94.80%
2023/01/08 13:39:26 2982919/3147639 94.77%
2023/01/09 13:39:26 2998712/3176934 94.39%
2023/01/10 13:39:26 2994631/3180844 94.15%
2023/01/13 02:12:23 3159307/3318531 95.20%
2023/01/14 02:12:23 3021483/3193402 94.62%
2023/01/15 02:12:23 3010165/3175348 94.80%
2023/01/16 02:12:23 3010367/3178627 94.71%
2023/01/17 02:12:23 2876650/3059836 94.01%
2023/01/18 02:12:23 2676927/2846637 94.04%
2023/01/19 02:12:23 2689500/2840457 94.69%
2023/01/20 02:12:23 2593768/2734524 94.85%
2023/01/21 02:12:23 2625426/2779702 94.45%
2023/01/22 02:12:23 2730913/2874371 95.01%
2023/01/23 02:12:23 2713914/2856540 95.01%
2023/01/24 02:12:23 2712147/2862818 94.74%
2023/01/25 02:12:23 2764751/2916888 94.78%
2023/01/26 02:12:23 2823173/2977070 94.83%
2023/01/27 02:12:23 2803899/2963868 94.60%
2023/01/28 02:12:23 2752513/2911332 94.54%
2023/01/29 02:12:23 2856561/3005275 95.05%
2023/01/30 02:12:23 2885433/3041352 94.87%
2023/01/31 02:12:23 2861213/3032729 94.34%
2023/02/01 02:12:23 2874864/3039998 94.57%
2023/02/02 02:12:23 2889754/3049058 94.78%
2023/02/03 02:12:23 2914695/3062367 95.18%
2023/02/04 02:12:23 2847896/2998863 94.97%
2023/02/05 02:12:23 2811298/2946670 95.41%
2023/02/06 02:12:23 2761140/2902851 95.12%
2023/02/07 02:12:23 2737250/2890104 94.71%
2023/02/08 02:12:23 2724894/2874121 94.81%
2023/02/09 02:12:23 2755421/2887012 95.44%
2023/02/10 02:12:23 2707386/2840938 95.30%
2023/02/11 02:12:23 2718711/2849491 95.41%
2023/02/12 02:12:23 2605462/2724865 95.62%
2023/02/13 02:12:23 2625173/2756602 95.23%
2023/02/14 02:12:23 2580373/2714888 95.05%
2023/02/15 02:12:23 2603552/2734293 95.22%
2023/02/16 02:12:23 2597875/2736427 94.94%
2023/02/17 02:12:23 2603329/2731863 95.30%
2023/02/18 03:06:58 2783076/2909641 95.65%
2023/02/19 03:06:58 2727879/2848886 95.75%
2023/02/20 03:06:58 2560371/2677627 95.62%
2023/02/21 03:06:58 2580350/2712691 95.12%
2023/02/22 03:06:58 2388451/2546924 93.78%
2023/02/23 03:06:58 2083081/2257625 92.27%
2023/02/24 03:06:58 2146725/2330121 92.13%
2023/02/25 03:06:58 2202918/2391555 92.11%
2023/02/26 03:06:58 2164685/2335185 92.70%
2023/02/27 03:06:58 2136689/2326754 91.83%
2023/02/28 03:06:58 2093819/2302734 90.93%
2023/03/01 03:06:58 2079724/2292969 90.70%
2023/03/02 03:06:58 2024533/2237584 90.48%
2023/03/03 03:06:58 1969091/2164831 90.96%
2023/03/04 03:06:58 1944731/2111440 92.10%
2023/03/05 03:06:58 1911220/2063561 92.62%
2023/03/06 03:06:58 1837312/1997782 91.97%
2023/03/07 03:06:58 1726116/1894914 91.09%
2023/03/08 03:06:58 1739224/1911262 91.00%
2023/03/09 03:06:58 1753215/1913511 91.62%
2023/03/10 03:06:58 1789072/1970817 90.78%
2023/03/11 03:06:58 1761480/1941561 90.72%
2023/03/12 03:06:58 1757541/1924281 91.33%
2023/03/13 03:06:58 1696244/1868432 90.78%
2023/03/14 20:17:54 752394/834628 90.15%
2023/03/15 20:17:54 756717/841520 89.92%
2023/03/16 20:17:54 758070/841276 90.11%
2023/03/17 20:17:54 752392/831973 90.43%
2023/03/18 20:17:54 763324/838603 91.02%
2023/03/19 20:17:54 755424/832243 90.77%
2023/03/20 20:17:54 757856/835657 90.69%
2023/03/21 20:17:54 858242/948627 90.47%
2023/03/23 03:25:05 866883/961001 90.21%
2023/03/24 03:25:05 854870/950286 89.96%
2023/03/25 03:25:05 857745/946211 90.65%
2023/03/26 03:25:05 876268/964114 90.89%
2023/03/27 03:25:05 892848/983992 90.74%
2023/03/28 03:25:05 872174/976084 89.35%
2023/03/29 03:25:05 871535/971938 89.67%
```
</details>
Note that this is not the same as ["no address in clientID-to-IP map"](tpo/anti-censorship/pluggable-transports/snowflake#40084):
that warning is logged when a `client_ip` was present,
but the server could not cache it long enough to associate it with a session.
This is different: `client_ip` is just not present.David Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/webtunnel/-/issues/18Create a container image for webtunnel2023-04-19T14:33:45Zmeskiomeskio@torproject.orgCreate a container image for webtunnelWebtunnels bridge operators might want to be able to use a container to run their bridge.
We should produce some documentation on how to run the container.Webtunnels bridge operators might want to be able to use a container to run their bridge.
We should produce some documentation on how to run the container.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetshelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40066BridgeDB requires outdated packages with known CVEs2023-10-12T14:24:01Zmeskiomeskio@torproject.orgBridgeDB requires outdated packages with known CVEsSelected vulnerable packages:
* Twisted 21.7.0
* Mechanize 0.4.5
* Pillow 8.2.0
* Werkzeug 2.2.2Selected vulnerable packages:
* Twisted 21.7.0
* Mechanize 0.4.5
* Pillow 8.2.0
* Werkzeug 2.2.2meskiomeskio@torproject.orgmeskiomeskio@torproject.org2023-05-31https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/124Design a user interface for Lox2023-03-27T18:54:21ZPhilipp Winterphw@torproject.orgDesign a user interface for LoxWe need to start thinking about Lox's user interface. In an ideal world, we would build it into Tor Browser and make it so that users will never be exposed to bridge lines at all.
For now, Salmon supports the following user-facing actio...We need to start thinking about Lox's user interface. In an ideal world, we would build it into Tor Browser and make it so that users will never be exposed to bridge lines at all.
For now, Salmon supports the following user-facing actions:
1. Sign up by being invited by somebody else.
2. Sign up by registering.
3. Request a new bridge.
4. Invite somebody else to Lox.
5. (Optional) Take a look at your reputationSponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibethttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/webtunnel/-/issues/17improve descriptor arguments published2023-10-24T08:17:20Zmeskiomeskio@torproject.orgimprove descriptor arguments publishedThis is the extra-info line I see for the bridge documented in the README:
```
transport webtunnel 127.0.0.1:11000 domainname=n97jlbvjwy3iotetc0axrtu3mm7gq1gajqv7-80-78-24-44.sslip.io,path=939e42a2-8f3f-4645-9d7b-8b9be28b157e,port=443,pu...This is the extra-info line I see for the bridge documented in the README:
```
transport webtunnel 127.0.0.1:11000 domainname=n97jlbvjwy3iotetc0axrtu3mm7gq1gajqv7-80-78-24-44.sslip.io,path=939e42a2-8f3f-4645-9d7b-8b9be28b157e,port=443,publicVersion=0.0.1
```
Currently webtunnel is publishing the localhost IP and port where is listening for connections from the http proxy, we might want to publish a different IP address and port there as they will be used to build the bridgeline. The README uses 192.0.3.2:1, but AFAIK we can't use the same IP-port combination for several bridges as tor will consider is the same bridge and ignore the others. We could generate a random IP address from a private range, but maybe is better to put there the public IP address of the domain we are using as front. Any ideas?
I see in the extra-info line `domainname` and `path`, where AFAIK the client expects `url`. I think this is not coming from the code itself, but from `ServerTransportOptions` in torrc. Isn't it? Maybe we can improve the README to document how that is configured to include the `url` option (and fix our public tunnel).
I have some doubts about `publicVersion`, I think is weird to keep that in the bridgeline. rdsys could strip it out, but is pretty hacky. I would love to see https://gitlab.torproject.org/tpo/core/tor/-/issues/11101 solving that problem instead andSponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetshelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/122Update x/net and x/crypto to latest version in Obfs4proxy and Snowflake2023-04-20T08:36:45ZtlaUpdate x/net and x/crypto to latest version in Obfs4proxy and SnowflakeSomebody sent me an issue about this:
https://github.com/tladesignz/IPtProxy/issues/45
> For security purposes, golang.org/x/crypto and golang.org/x/net should be updated to latest versions. This also requires the obfs4 and snowflake s...Somebody sent me an issue about this:
https://github.com/tladesignz/IPtProxy/issues/45
> For security purposes, golang.org/x/crypto and golang.org/x/net should be updated to latest versions. This also requires the obfs4 and snowflake submodules to do the same thing, so there needs to be some coordination between all three projects for this to happen.
Please let me know, if this is valid, and if there's any problems with that request! Thank you!meskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/121migrate our images outside docker hub2024-03-04T09:50:16Zmeskiomeskio@torproject.orgmigrate our images outside docker hubWe got an email saying that April 14th our [thetorproject](https://hub.docker.com/u/thetorproject) organization will be close. There is a [FAQ](https://web.docker.com/rs/790-SSB-375/images/privatereposfaq.pdf) hidden in their website wit...We got an email saying that April 14th our [thetorproject](https://hub.docker.com/u/thetorproject) organization will be close. There is a [FAQ](https://web.docker.com/rs/790-SSB-375/images/privatereposfaq.pdf) hidden in their website with some information about it.
If we don't do anything our images will deleted April 14th, making our 900k+ unable to update them or to install them. But docker promises that *your organization’s namespace will not be released, so other users can’t “squat” your images*.
We can ask to be included in the [Docker-Sponsored Open Source Program](https://www.docker.com/community/open-source/application/), in that case they will *defer any organization suspension or deletion while DSOS application is under review, and give organizations at least 30 days before we suspend the organization if the application is ultimately rejected*. We should qualify for it, but there are many stories in internet of people being rejected from it.
We can move to another registry, there are few options:
* TPA plans to add a registry (https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/89)
* [quay.io](https://quay.io/)
* [github](https://docs.github.com/en/packages/working-with-a-github-packages-registry/working-with-the-container-registry)
* An individual docker hub account, will require changing the namemeskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40265Standalone proxy - error polling probe - 'certificate is not standards compli...2023-03-16T17:40:24ZMarkCStandalone proxy - error polling probe - 'certificate is not standards compliant'@meskio As requested.
Yesterday I updated one of my proxies after !136 completed. When I run the proxy it starts, then begins probetest, then immediately (and repeatedly) returns an error:
2023/03/16 00:45:47 error polling probe: x509:...@meskio As requested.
Yesterday I updated one of my proxies after !136 completed. When I run the proxy it starts, then begins probetest, then immediately (and repeatedly) returns an error:
2023/03/16 00:45:47 error polling probe: x509: “snowflake-broker.torproject.net” certificate is not standards compliant
2023/03/16 00:45:47 NAT type: unknown
2023/03/16 00:45:48 error polling broker: x509: “snowflake-broker.torproject.net” certificate is not standards compliant
2023/03/16 00:45:48 Error reading broker response: unexpected end of JSON input
2023/03/16 00:45:48 body:
2023/03/16 00:45:48 bad offer from broker
Here’s the log of the start up (note that the git pull in this sequence says 'already up to date' because I ran it a second time as a double check).
[proxy_error_polling_broker_-_03_15_23.txt](/uploads/e8d343dc26e784d66e59880530374774/proxy_error_polling_broker_-_03_15_23.txt)
On the previous build everything was functioning properly.
@itchyonion Might this result be caused by the bug you mentioned in #40108 yesterday?itchyonionitchyonionhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40262Deploy snowflake-server for QueuePacketConn buffer reuse fix (#40260)2023-07-18T00:47:30ZDavid Fifielddcf@torproject.orgDeploy snowflake-server for QueuePacketConn buffer reuse fix (#40260)#40260 describes a buffer reuse error in `QueuePacketConn` resulting from #40199.
!140 promises to fix it.
After merging !140, deploy to the bridges and restart.
- [x] snowflake-01
- [x] snowflake-02
/cc @cohosh @linus#40260 describes a buffer reuse error in `QueuePacketConn` resulting from #40199.
!140 promises to fix it.
After merging !140, deploy to the bridges and restart.
- [x] snowflake-01
- [x] snowflake-02
/cc @cohosh @linusDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/156Missing Authentication at Resource Registration2023-10-12T14:26:44Zmeskiomeskio@torproject.orgMissing Authentication at Resource RegistrationWhile auditing the source code of the Rdsys, it is noticed that the Rdsys backend has missing authentication for the resource registration endpoint. This allows an adversary to register arbitrary malicious resources which will be distrib...While auditing the source code of the Rdsys, it is noticed that the Rdsys backend has missing authentication for the resource registration endpoint. This allows an adversary to register arbitrary malicious resources which will be distributed to the users.
The following snippet of code shows the affected code where the resource registration endpoint does not have any kind of authentication checks as compared to the other endpoints.
Affected file:
rdsys/internal/backend.go
Affected code:
```go
func (b *BackendContext) resourcesHandler(w http.ResponseWriter, r *http.Request) {
switch r.Method {
[...]
case http.MethodPost:
if r.URL.Path == b.Config.Backend.ResourcesEndpoint {
b.postResourcesHandler(w, r)
[...]
}
func (b *BackendContext) postResourcesHandler(w http.ResponseWriter, req *http.Request) {
body, err := ioutil.ReadAll(req.Body)
[...]
rTypes := map[string]struct{}{}
for _, r := range rs {
b.Resources.Add(r)
rTypes[r.Type()] = struct{}{}
log.Printf("Added %s's %q resource to collection.", req.RemoteAddr, r.Type())
}
for rType := range rTypes {
b.rStore.Save(rType)
}
[...]
}
```
The aforementioned issue can be reproduced by executing the following cURL request.
PoC:
```
curl http://localhost:7100/resources -i --data '[{"type": "obfs2", "address": "1.2.3.2", "port": 1235, "fingerprint":"10282810115283F99ADE5CFE42D49644F45D715D"}]' -XPOST
# Other endpoints which are missing authentication
curl -i -s -k -X $'GET' \
-H $'Host: localhost:7100' \
$'http://localhost:7100/status?id=10282810115283F99ADE5CFE42D49644F45D715D'
curl -i -s -k -X $'GET' \
-H $'Host: localhost:7100' \
$'http://localhost:7100/rdsys-backend-metrics'
```
To address the issue, it is strongly advised to implement robust authentication mechanisms for all endpoints, with particular attention paid to the registration of resources. This will help to ensure that only authorized users are able to register resources, thereby reducing the risk of unauthorized access.meskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/154can we distribute bridges with unrechable ORPort?2023-08-23T18:59:04Zmeskiomeskio@torproject.orgcan we distribute bridges with unrechable ORPort?AFAIK if a bridge doesn't have the ORPort public the bridge authority will don't add the 'Running' flag, but will still add the bridge to the bridge descriptors. Is that correct?
rdsys currently doesn't distribute bridges that don't hav...AFAIK if a bridge doesn't have the ORPort public the bridge authority will don't add the 'Running' flag, but will still add the bridge to the bridge descriptors. Is that correct?
rdsys currently doesn't distribute bridges that don't have the 'Running' flag. Maybe if we start ignoring the 'Running' flag, and just use bridgestrap to check if a bridge is working, we can allow bridge operators to publish bridges with only the PT port accessible.
If so we could close https://gitlab.torproject.org/tpo/core/tor/-/issues/7349meskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40065HTTPS CAPTCHA challenge doesn't work with new frontend2023-03-07T21:27:09ZCecylia BocovichHTTPS CAPTCHA challenge doesn't work with new frontendWhile debugging #40064, I noticed that the new `captcha.html` template is missing two necessary fields to pass the CAPTCHA challenge to the BridgeDB backend (see https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40064#n...While debugging #40064, I noticed that the new `captcha.html` template is missing two necessary fields to pass the CAPTCHA challenge to the BridgeDB backend (see https://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40064#note_2884130)
Without the right input names in the captcha challenge submission form, BridgeDB doesn't know where to look to verify the challenge and solution.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/bridgedb/-/issues/40064BridgeDB https and moat distributors down after upgrade2023-03-07T21:27:00ZCecylia BocovichBridgeDB https and moat distributors down after upgradeAfter the bridgedb upgrade this morning, both the https and moat distributors appear to have stopped working.After the bridgedb upgrade this morning, both the https and moat distributors appear to have stopped working.https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/153give only obfs4 bridges to onbasca2023-03-14T14:34:22Zmeskiomeskio@torproject.orggive only obfs4 bridges to onbascaOnbasca is unable to test vanilla bridges, let's give only obfs4 bridges to onbasca.Onbasca is unable to test vanilla bridges, let's give only obfs4 bridges to onbasca.meskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40255snowflake-02: increase number of tor instances from 4 to 122023-02-17T03:15:20ZDavid Fifielddcf@torproject.orgsnowflake-02: increase number of tor instances from 4 to 12The idea is that snowflake-02 will soon be handling a roughly equal amount of traffic as snowflake-01, which currently runs 12 instances.The idea is that snowflake-02 will soon be handling a roughly equal amount of traffic as snowflake-01, which currently runs 12 instances.David Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40253Restart snowflake bridges for haproxy CVE-2023-0056, CVE-2023-257252023-05-25T15:04:32ZDavid Fifielddcf@torproject.orgRestart snowflake bridges for haproxy CVE-2023-0056, CVE-2023-25725https://security-tracker.debian.org/tracker/DSA-5348-1
https://lists.debian.org/debian-security-announce/2023/msg00037.html
> Two vulnerabilities were discovered in HAProxy, a fast and reliable load
> balancing reverse proxy, which may ...https://security-tracker.debian.org/tracker/DSA-5348-1
https://lists.debian.org/debian-security-announce/2023/msg00037.html
> Two vulnerabilities were discovered in HAProxy, a fast and reliable load
> balancing reverse proxy, which may result in denial of service, or
> bypass of access controls and routing rules via specially crafted
> requests.
>
> For the stable distribution (bullseye), these problems have been fixed in
> version 2.2.9-2+deb11u4.
>
> We recommend that you upgrade your haproxy packages.
https://ubuntu.com/security/notices/USN-5869-1
> HAProxy could allow unintended access to network services.
>
> The problem can be corrected by updating your system to the following package versions:
>
> Ubuntu 20.04
>
> [haproxy - 2.0.29-0ubuntu1.3](https://launchpad.net/ubuntu/+source/haproxy/2.0.29-0ubuntu1.3)
- [x] snowflake-01
- [x] snowflake-02
/cc @linusDavid Fifielddcf@torproject.orgDavid Fifielddcf@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40252Standalone proxy probetest Wiki inconsistency and implementation details2023-03-07T18:13:49ZitchyonionStandalone proxy probetest Wiki inconsistency and implementation details1. Our [Wiki](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/wikis/NAT-matching#determining-nat-behaviour) states that:
> We determine the NAT behaviour of clients by using the tricks in [RFC 5780](ht...1. Our [Wiki](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/wikis/NAT-matching#determining-nat-behaviour) states that:
> We determine the NAT behaviour of clients by using the tricks in [RFC 5780](https://tools.ietf.org/html/rfc5780) ... For standalone proxies written in Go, we use the same method.
Which is not true since we switched to probetest https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/commit/00f8f85f412878c2066fcb5d3f4739e50912a925
Cecylia linked the issue for the change: https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40013. I will update the Wiki.
2. Right now we are [logging the SDP offer](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/main/proxy/lib/snowflake.go#L676) used for probetest, which might be misleading because before looking into this in detail I always thought we were logging the SDP candidates used for WebRTC connection. Does this help the user in any way or is simplying logging the resultant NAT type enough? Some options to consider:
- keep the same logging, but made it extra clear that this is only used for probetest, not peer connection
- log SDP candidates for WebRTC. I think this would be helpful for debugging, but could produce much more logs
- log both
- [x] Update Wiki with reason to use probetest
- [x] Research on whether we should respect proxy options in probetest
- [x] Decide what to logitchyonionitchyonionhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake-webext/-/issues/80security: tighten CSP (`connect-src`)2023-04-17T19:06:22ZWofWcawofwca@protonmail.comsecurity: tighten CSP (`connect-src`)Besides WebRTC, currently we only connect to the broker and the server, so only allowing the extension to connect to those servers should improve security.
Later, if/when we allow to connect to any address (e.g. https://gitlab.torproject...Besides WebRTC, currently we only connect to the broker and the server, so only allowing the extension to connect to those servers should improve security.
Later, if/when we allow to connect to any address (e.g. https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40166), we may keep the CSP to only allow `ws:`/`wss:` connections, and `https://broker...`.
I need to remind that currently the `connect-src` directive doesn't affect WebRTC. See [1](https://github.com/sandstormports/community-project/issues/12), [2](https://github.com/w3c/webappsec-csp/issues/92).
Also need to keep configurability in mind, i.e. if users want to switch to a custom broker/relay.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/conjure/-/issues/25Feedback: successful conjure test on tbb nightly2023-03-07T21:15:24ZMarkCFeedback: successful conjure test on tbb nightly@cohosh At wkrp’s suggestion on net4people I tested conjure on tbb nightly. It bootstraped successfully. I’ll confess that I didn’t follow all the instructions, just downloaded the nightly on my macos test machine, pasted in the bridgeli...@cohosh At wkrp’s suggestion on net4people I tested conjure on tbb nightly. It bootstraped successfully. I’ll confess that I didn’t follow all the instructions, just downloaded the nightly on my macos test machine, pasted in the bridgeline and connected. Tor log attached. Thx.[Conjure_test_success.rtf](/uploads/4c9501d3e6000275bc2850ccf4d3665c/Conjure_test_success.rtf)https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/117Make bridge lines address into 'human-memorable' address or easier for users ...2023-05-18T11:18:05ZGusMake bridge lines address into 'human-memorable' address or easier for users to copy/shareA frequent problem in our user support channels is users failing to add a new bridge due to various reasons:
- Sometimes they just copy a piece of it (IP:Port) (https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40552)
...A frequent problem in our user support channels is users failing to add a new bridge due to various reasons:
- Sometimes they just copy a piece of it (IP:Port) (https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/40552)
- In other cases they were adding more than one bridge without the right space/delimiter and then Tor fails to parse it right.
There are other instances where these poorly formatted addresses result in the user being unable to connect to Tor and lead them to report that the bridge is blocked in their country. (cc: @nina @championquizzer)
This is affecting users especially from high censored regions because we're sharing private bridges with them as other pluggable transports didn't work for them.
Maybe qr-codes could be used with bridgemojis?
Maybe bridges with human-memorables address would make this easier?meskiomeskio@torproject.orgmeskiomeskio@torproject.org