Snowflake issueshttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues2023-03-07T15:49:08Zhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40254Nil Pointer Crash when Initializing Snowflake Proxy2023-03-07T15:49:08ZbimNil Pointer Crash when Initializing Snowflake Proxyhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/main/proxy/lib/snowflake.go#L568
Line 568 ought to be moved below 589 - if the event dispatcher isn't set the proxy will crash. I came across this b...https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/main/proxy/lib/snowflake.go#L568
Line 568 ought to be moved below 589 - if the event dispatcher isn't set the proxy will crash. I came across this bumping snowflake to the the latest release in Orbot via our IPtProxy wrapper library.
https://github.com/tladesignz/IPtProxy/issues/39
For now, we simply just init'd our own event dispatcher instance to sidestep the crash.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40220Close stale connections in standalone proxy2022-11-29T14:20:34ZCecylia BocovichClose stale connections in standalone proxyWe've received several reports (#40211) of standalone proxies that have long-lived connections with clients but zero bytes transferred. The browser-based snowflake proxies (i.e., the web extension and badge) have a [timeout in place to c...We've received several reports (#40211) of standalone proxies that have long-lived connections with clients but zero bytes transferred. The browser-based snowflake proxies (i.e., the web extension and badge) have a [timeout in place to close stale connections](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake-webext/-/blob/29a4cc6e0970f2e10ed610b8ae8449eafe75472c/proxypair.js#L163) after [30 seconds](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake-webext/-/blob/29a4cc6e0970f2e10ed610b8ae8449eafe75472c/config.js#L38) of inactivity. This aligns with a [client-side timeout](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/ac8562803ab9621d037bd1b3710c59799c7aa6d5/client/lib/snowflake.go#L49) that closes stale connections to proxies after 20s of inactivity.
It's possible that the long-lived connections these standalone proxies are seeing are from clients not using our snowflake client code. Or that the client-side closures are not being received by the proxies. In any case, we should add an inactivity timeout to the standalone proxies to try and clean up these connections and free up resources in a similar way that the browser-based proxies do.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40206Broker status 500 when client requests fingerprint of snowflake-022023-03-14T16:42:43ZDavid Fifielddcf@torproject.orgBroker status 500 when client requests fingerprint of snowflake-02I was not sure whether distributed bridge support
(#40129, #28651) was supposed to be working yet, so I tried it.
I am using the client from commit 9ce1de4eee4e23c918c7c5e96666ff5c6ddc654e.
When I tried rendezvousing with the fingerprint...I was not sure whether distributed bridge support
(#40129, #28651) was supposed to be working yet, so I tried it.
I am using the client from commit 9ce1de4eee4e23c918c7c5e96666ff5c6ddc654e.
When I tried rendezvousing with the fingerprint of the
[snowflake-02 bridge](tpo/anti-censorship/pluggable-transports/snowflake#40122),
I got a 500 Internal Server Error, which was unexpected.
It makes me wonder if the broker is encountering an internal error and panicking,
or if 500 is the intended response code.
This torrc file works:
```
UseBridges 1
SocksPort auto
DataDirectory datadir
ClientTransportPlugin snowflake exec ./client -log snowflake.log
Bridge snowflake 192.0.2.3:80 2B280B23E1107BB62ABFC40DDCC8824814F80A72 fingerprint=2B280B23E1107BB62ABFC40DDCC8824814F80A72 url=https://snowflake-broker.torproject.net/ ice=stun:stun.voip.blackberry.com:3478,stun:stun.altar.com.pl:3478,stun:stun.antisip.com:3478,stun:stun.bluesip.net:3478,stun:stun.dus.net:3478,stun:stun.epygi.com:3478,stun:stun.sonetel.com:3478,stun:stun.sonetel.net:3478,stun:stun.stunprotocol.org:3478,stun:stun.uls.co.za:3478,stun:stun.voipgate.com:3478,stun:stun.voys.nl:3478
```
This torrc file does not work. The only difference is changing the fingerprint
`2B280B23E1107BB62ABFC40DDCC8824814F80A72` to `8838024498816A039FCBBAB14E6F40A0843051FA`
in both places.
```
UseBridges 1
SocksPort auto
DataDirectory datadir
ClientTransportPlugin snowflake exec ./client -log snowflake.log
Bridge snowflake 192.0.2.3:80 8838024498816A039FCBBAB14E6F40A0843051FA fingerprint=8838024498816A039FCBBAB14E6F40A0843051FA url=https://snowflake-broker.torproject.net/ ice=stun:stun.voip.blackberry.com:3478,stun:stun.altar.com.pl:3478,stun:stun.antisip.com:3478,stun:stun.bluesip.net:3478,stun:stun.dus.net:3478,stun:stun.epygi.com:3478,stun:stun.sonetel.com:3478,stun:stun.sonetel.net:3478,stun:stun.stunprotocol.org:3478,stun:stun.uls.co.za:3478,stun:stun.voipgate.com:3478,stun:stun.voys.nl:3478
```
The broker's HTTP response is:
```
500 Internal Server Error
Access-Control-Allow-Headers: Origin, X-Session-ID
Access-Control-Allow-Origin: *
Content-Length: 0
Date: Tue, 04 Oct 2022 16:33:06 GMT
```https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40135Cannot build snowflake/proxy2022-07-09T04:20:15ZcypherpunksCannot build snowflake/proxyHello! I want to install the snowflake proxy on my raspberry pi, but I can't seem to build it. I updated my system via apt-get update/update succesfully and the installation of git and golang were also successful. But after cloning the ...Hello! I want to install the snowflake proxy on my raspberry pi, but I can't seem to build it. I updated my system via apt-get update/update succesfully and the installation of git and golang were also successful. But after cloning the git, I went to /home/pi/snowflake/proxy and when I enter go build the system gives me this:
```
pi@raspberrypi:~/snowflake/proxy $ go build
main.go:5:2: cannot find package "git.torproject.org/pluggable-transports/snowflake.git/v2/common/event" in any of:
/usr/lib/go-1.7/src/git.torproject.org/pluggable-transports/snowflake.git/v2/common/event (from $GOROOT)
($GOPATH not set)
main.go:12:2: cannot find package "git.torproject.org/pluggable-transports/snowflake.git/v2/common/safelog" in any of:
/usr/lib/go-1.7/src/git.torproject.org/pluggable-transports/snowflake.git/v2/common/safelog (from $GOROOT)
($GOPATH not set)
main.go:13:2: cannot find package "git.torproject.org/pluggable-transports/snowflake.git/v2/proxy/lib" in any of:
/usr/lib/go-1.7/src/git.torproject.org/pluggable-transports/snowflake.git/v2/proxy/lib (from $GOROOT)
($GOPATH not set)
```
What is the problem here, and how can I fix it? Whe I look into /usr/lib/go-1.7/src/ there is no git.torproject.org subdirectory
I hope, you can help me.itchyonionitchyonionhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40103mix of past and present in snowflake proxy log2022-07-09T04:20:15Ztoralfmix of past and present in snowflake proxy log"In the last 1h0m0s, there are 28 connections. Traffic Relayed ↑ 273 MB, ↓ 273 MB."
are -> were
or ?"In the last 1h0m0s, there are 28 connections. Traffic Relayed ↑ 273 MB, ↓ 273 MB."
are -> were
or ?https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40101Failures at standalone proxy2023-05-16T20:13:44ZCecylia BocovichFailures at standalone proxyWe got the following error log from a proxy volunteer:
```
2022/02/11 00:38:05 In the last 1h0m0s, there are 103 connections. Traffic Relayed ↑ 2 GB, ↓ 2 GB.
ortc ERROR: 2022/02/11 02:28:16 Failed to accept data channel: failed to se...We got the following error log from a proxy volunteer:
```
2022/02/11 00:38:05 In the last 1h0m0s, there are 103 connections. Traffic Relayed ↑ 2 GB, ↓ 2 GB.
ortc ERROR: 2022/02/11 02:28:16 Failed to accept data channel: failed to send ChannelOpen ACK: sending payload data in non-established state: state=Closed
ortc ERROR: 2022/02/11 02:29:25 Failed to accept data channel: failed to send ChannelOpen ACK: sending payload data in non-established state: state=Closed
2022/02/11 01:38:05 In the last 1h0m0s, there are 66 connections. Traffic Relayed ↑ 1 GB, ↓ 1 GB.
2022/02/11 02:38:05 In the last 1h0m0s, there are 74 connections. Traffic Relayed ↑ 560 MB, ↓ 560 MB.
2022/02/11 03:38:05 In the last 1h0m0s, there are 69 connections. Traffic Relayed ↑ 5 GB, ↓ 5 GB.
2022/02/11 04:38:05 In the last 1h0m0s, there are 68 connections. Traffic Relayed ↑ 1 GB, ↓ 1 GB.
2022/02/11 05:38:05 In the last 1h0m0s, there are 86 connections. Traffic Relayed ↑ 2 GB, ↓ 2 GB.
2022/02/11 06:38:05 In the last 1h0m0s, there are 91 connections. Traffic Relayed ↑ 2 GB, ↓ 2 GB.
2022/02/11 07:38:05 In the last 1h0m0s, there are 111 connections. Traffic Relayed ↑ 613 MB, ↓ 613 MB.
sctp ERROR: 2022/02/11 08:53:20 [0xc073c969c0] stream 1 not found)
sctp ERROR: 2022/02/11 08:53:20 [0xc073c969c0] stream 1 not found)
sctp ERROR: 2022/02/11 09:03:19 [0xc0a7166340] stream 1 not found)
ortc ERROR: 2022/02/11 09:15:45 Failed to accept data channel: failed to send ChannelOpen ACK: sending payload data in non-established state: state=Closed
sctp ERROR: 2022/02/11 09:37:53 [0xc0103c36c0] stream 1 not found)
sctp ERROR: 2022/02/11 09:37:53 [0xc0103c36c0] stream 1 not found)
2022/02/11 08:38:05 In the last 1h0m0s, there are 121 connections. Traffic Relayed ↑ 212 MB, ↓ 212 MB.
ortc ERROR: 2022/02/11 09:41:30 Failed to accept data channel: failed to send ChannelOpen ACK: sending payload data in non-established state: state=Closed
sctp ERROR: 2022/02/11 09:54:55 [0xc02e8e1860] stream 1 not found)
sctp ERROR: 2022/02/11 09:54:55 [0xc02e8e1860] stream 1 not found)
sctp ERROR: 2022/02/11 09:54:55 [0xc02e8e1860] stream 1 not found)
sctp ERROR: 2022/02/11 09:54:55 [0xc02e8e1860] stream 1 not found)
sctp ERROR: 2022/02/11 09:54:58 [0xc073d15380] stream 1 not found)
sctp ERROR: 2022/02/11 09:55:01 [0xc0103c29c0] stream 1 not found)
sctp ERROR: 2022/02/11 09:55:03 [0xc003828ea0] stream 1 not found)
sctp ERROR: 2022/02/11 09:55:03 [0xc003828ea0] stream 1 not found)
ortc ERROR: 2022/02/11 10:27:42 Failed to accept data channel: failed to send ChannelOpen ACK: sending payload data in non-established state: state=Closed
Killed
(by kernel OOM killer.)
```shelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40098snowflake_server.httpHandler.ln is not initialized, leading to panic in onesh...2022-07-09T04:20:16ZDavid Fifielddcf@torproject.orgsnowflake_server.httpHandler.ln is not initialized, leading to panic in oneshotModeThere are a few of these in the snowflake-server log:
```
2022/02/01 16:44:11 http: panic serving [scrubbed]: runtime error: invalid memory address or nil pointer dereference
goroutine 2513699 [running]:
net/http.(*conn).serve.func1(0xc...There are a few of these in the snowflake-server log:
```
2022/02/01 16:44:11 http: panic serving [scrubbed]: runtime error: invalid memory address or nil pointer dereference
goroutine 2513699 [running]:
net/http.(*conn).serve.func1(0xc007576dc0)
/usr/lib/go-1.15/src/net/http/server.go:1801 +0x147
panic(0x7ea7a0, 0xb0c450)
/usr/lib/go-1.15/src/runtime/panic.go:975 +0x47a
git.torproject.org/pluggable-transports/snowflake.git/v2/server/lib.(*SnowflakeListener).queueConn(0x0, 0x8dc820, 0xc04b943740, 0xc03557fbb0, 0xc04b9436e0)
./snowflake/server/lib/snowflake.go:275 +0x37
git.torproject.org/pluggable-transports/snowflake.git/v2/server/lib.oneshotMode(...)
./snowflake/server/lib/http.go:117
git.torproject.org/pluggable-transports/snowflake.git/v2/server/lib.(*httpHandler).ServeHTTP(0xc0001a0e80, 0x8d9800, 0xc057a75880, 0xc0dda2cd00)
./snowflake/server/lib/http.go:105 +0x5b7
net/http.serverHandler.ServeHTTP(0xc0001ee0e0, 0x8d9800, 0xc057a75880, 0xc0dda2cd00)
/usr/lib/go-1.15/src/net/http/server.go:2843 +0xa3
net/http.(*conn).serve(0xc007576dc0, 0x8da1c0, 0xc013b86b00)
/usr/lib/go-1.15/src/net/http/server.go:1925 +0x8ad
created by net/http.(*Server).Serve
/usr/lib/go-1.15/src/net/http/server.go:2969 +0x36c
```
`oneshotMode` is called with `httpHandler.ln` as the `*SnowflakeListener` argument:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/e828b0607662c7325f4d1bbf4c5072ce81f38fb9/server/lib/http.go#L100-105
```go
// We didn't find a matching token, which means that we are
// dealing with a client that doesn't know about such things.
// "Unread" the token by constructing a new Reader and pass it
// to the old one-session-per-WebSocket mode.
conn2 := &overrideReadConn{Conn: conn, Reader: io.MultiReader(bytes.NewReader(token[:]), conn)}
err = oneshotMode(conn2, addr, handler.ln)
```
`httpHandler` is:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/e828b0607662c7325f4d1bbf4c5072ce81f38fb9/server/lib/http.go#L63-68
```go
type httpHandler struct {
// pconn is the adapter layer between stream-oriented WebSocket
// connections and the packet-oriented KCP layer.
pconn *turbotunnel.QueuePacketConn
ln *SnowflakeListener
}
```
But in the only place `httpHandler` is instantiated, its `ln` field is left uninitialized:
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/e828b0607662c7325f4d1bbf4c5072ce81f38fb9/server/lib/snowflake.go#L80-85
```go
handler := httpHandler{
// pconn is shared among all connections to this server. It
// overlays packet-based client sessions on top of ephemeral
// WebSocket connections.
pconn: turbotunnel.NewQueuePacketConn(addr, clientMapTimeout),
}
```Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40088Snowflake server keeps failing unexpectedly2022-07-09T04:20:46ZCecylia BocovichSnowflake server keeps failing unexpectedlyThe snowflake server has failed twice now in the last two weeks.
[After the first failure](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40060#note_2767868), we upgraded the capacity of the Sn...The snowflake server has failed twice now in the last two weeks.
[After the first failure](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40060#note_2767868), we upgraded the capacity of the Snowflake bridge (#40085).
Today when it failed we received an alert from monit: https://lists.torproject.org/pipermail/anti-censorship-alerts/2021-December/000786.html
Unfortunately, in my rush to fix it I didn't check the CPU usage :/ let's use this ticket to track the issue and note what we notice the next time it occurs.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibetshelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40087Let's Encrypt "DST Root X3" root expiration affects old Android clients acces...2024-01-22T16:33:10ZcybertaLet's Encrypt "DST Root X3" root expiration affects old Android clients accessing brokerRunning Snowflake with the default config mentioned in this repository and shown below Snowflake fails to create a connection on some Android devices (apparently older Android versions, I could reproduce that issue using Android 4 and An...Running Snowflake with the default config mentioned in this repository and shown below Snowflake fails to create a connection on some Android devices (apparently older Android versions, I could reproduce that issue using Android 4 and Android 6 on a real device and on an emulator).
The error log tells me the cause of the connection failure is an expired certificate.
`WebRTC: x509: certificate has expired or is not yet valid: current time 2021-12-28T16:12:58Z is after 2021-09-30T14:01:15Z Retrying... `
Default config, I'm referring to:
```
snowflake-target https://snowflake-broker.torproject.net.global.prod.fastly.net/
snowflake-front cdn.sstatic.net
```
Using a different broker and domain-fronting I can work around the issue (config taken from https://github.com/cohosh/snowflake)
Could you please have a look at the broker / domain fronting setup or adapt the documentation here?Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40071Increase of "unknown" NAT assignments by probetest since 2021-10-252023-06-20T18:24:54ZDavid Fifielddcf@torproject.orgIncrease of "unknown" NAT assignments by probetest since 2021-10-25https://lists.torproject.org/pipermail/anti-censorship-team/2021-October/000197.html
> ...looking into the broker graphs there is something weird since 2 days. The number of proxies with 'unknown' type of nat has rised heavily at the sam...https://lists.torproject.org/pipermail/anti-censorship-team/2021-October/000197.html
> ...looking into the broker graphs there is something weird since 2 days. The number of proxies with 'unknown' type of nat has rised heavily at the same time the 'restricted' nat has gone down. There are long periods without idle proxies and many requests being denied of nat type uknown. It doesn't look like the proxy capacity has gone down, can it be something broken on the way we test the nat type?
It seems that something is going wrong with probetest. A past problem we had with probetest not functioning properly was #40039. Currently the probetest process is again using 100% CPU.
It is possible this is some kind of slow resource exhaustion, or it's possible that probetest is simply overloaded with the number of proxies we have currently. At the [2021-10-28 anti-censorship team meeting](http://meetbot.debian.net/tor-meeting/2021/tor-meeting.2021-10-28-16.00.log.html#l-51) we decided to restart probetest and watch it to see how quickly it returns to its failure state, in order to distinguish these two possibilities.shelikhooshelikhoohttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40064How to manage CPU saturation?2022-04-05T15:23:39ZJacobo NájeraHow to manage CPU saturation?Hi,
For several months I have been running various snowflake proxies with the standalone version. On two occasions I identified CPU saturation. I currently have a snowflake proxy that consumes between 90% and 100% CPU.
CPU usage
![Ca...Hi,
For several months I have been running various snowflake proxies with the standalone version. On two occasions I identified CPU saturation. I currently have a snowflake proxy that consumes between 90% and 100% CPU.
CPU usage
![Captura_de_pantalla_de_2021-08-29_22-08-45](/uploads/b6af9f8067d28e058559538e3d583c7e/Captura_de_pantalla_de_2021-08-29_22-08-45.png)
Connections
![Captura_de_pantalla_de_2021-08-30_23-28-50](/uploads/285a0f49645210e8beedb55b7e4d439d/Captura_de_pantalla_de_2021-08-30_23-28-50.png)
Software setup
- Docker image thetorproject/snowflake-proxy:latest
- Debian Buster
How to manage CPU saturation?
Thanks, Jacobohttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40058Snowflake embed refuses to work on Chromium?2022-07-09T04:20:16ZkoutsieSnowflake embed refuses to work on Chromium?Snowflake (embed) simply refuses to enable on Chromium and Chrome.
At first i thought it was one of my extensions but a couple of friends reported the inability to enable Snowflake's embed while stating that the web extension version wor...Snowflake (embed) simply refuses to enable on Chromium and Chrome.
At first i thought it was one of my extensions but a couple of friends reported the inability to enable Snowflake's embed while stating that the web extension version works fine.https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40057A weird issue about domain name resolution2022-03-01T15:55:27Zdebug996A weird issue about domain name resolutionDevelopers of Snowflake,
I have trouble to use the snowflake bridge to connect to Tor. To find out the main issue, I managed to collect the log of snowflake. (attached below)
It can be seen that the snowflake program tries to resolve h...Developers of Snowflake,
I have trouble to use the snowflake bridge to connect to Tor. To find out the main issue, I managed to collect the log of snowflake. (attached below)
It can be seen that the snowflake program tries to resolve host names from [::1]:53, but there is no dns server listening, nor is there a statement for [::1]:53 in /etc/resolv.conf .
```
--- Starting Snowflake Client ---
2021/07/24 07:12:11 Using ICE servers:
2021/07/24 07:12:11 url: stun:stun.sonetel.com:3478
2021/07/24 07:12:11 url: stun:stun.stunprotocol.org:3478
2021/07/24 07:12:11 url: stun:stun.dus.net:3478
2021/07/24 07:12:11 url: stun:stun.uls.co.za:3478
2021/07/24 07:12:11 url: stun:stun.voip.blackberry.com:3478
2021/07/24 07:12:11 url: stun:stun.sonetel.net:3478
2021/07/24 07:12:11 url: stun:stun.bluesip.net:3478
2021/07/24 07:12:11 Rendezvous using Broker at: https://snowflake-broker.torproject.net.global.prod.fastly.net/
2021/07/24 07:12:11 Domain fronting using: cdn.sstatic.net
2021/07/24 07:12:11 Started SOCKS listener at 127.0.0.1:39363.
2021/07/24 07:12:11 Error resolving address: lookup stun.sonetel.com on [::1]:53: read udp [::1]:60448->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error creating STUN connection: lookup stun.sonetel.com on [::1]:53: read udp [::1]:60448->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error resolving address: lookup stun.stunprotocol.org on [::1]:53: read udp [::1]:46030->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error creating STUN connection: lookup stun.stunprotocol.org on [::1]:53: read udp [::1]:46030->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error resolving address: lookup stun.dus.net on [::1]:53: read udp [::1]:46841->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error creating STUN connection: lookup stun.dus.net on [::1]:53: read udp [::1]:46841->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error resolving address: lookup stun.uls.co.za on [::1]:53: read udp [::1]:51689->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error creating STUN connection: lookup stun.uls.co.za on [::1]:53: read udp [::1]:51689->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error resolving address: lookup stun.voip.blackberry.com on [::1]:53: read udp [::1]:51608->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error creating STUN connection: lookup stun.voip.blackberry.com on [::1]:53: read udp [::1]:51608->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error resolving address: lookup stun.sonetel.net on [::1]:53: read udp [::1]:35863->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error creating STUN connection: lookup stun.sonetel.net on [::1]:53: read udp [::1]:35863->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error resolving address: lookup stun.bluesip.net on [::1]:53: read udp [::1]:42763->[::1]:53: read: connection refused
2021/07/24 07:12:11 Error creating STUN connection: lookup stun.bluesip.net on [::1]:53: read udp [::1]:42763->[::1]:53: read: connection refused
2021/07/24 07:12:11 NAT Type: unknown
2021/07/24 07:12:11 SOCKS accepted: {192.0.2.3:1 map[]}
2021/07/24 07:12:11 ---- SnowflakeConn: begin collecting snowflakes ---
2021/07/24 07:12:11 ---- SnowflakeConn: starting a new session ---
2021/07/24 07:12:11 ---- SnowflakeConn: begin stream 3 ---
2021/07/24 07:12:11 redialing on same connection
2021/07/24 07:12:11 WebRTC: Collecting a new Snowflake. Currently at [0/1]
2021/07/24 07:12:11 snowflake-c33c6e11b0e8bdf9 connecting...
2021/07/24 07:12:11 WebRTC: DataChannel created.
2021/07/24 07:12:11 WebRTC: Created offer
2021/07/24 07:12:11 WebRTC: Set local description
2021/07/24 07:12:11 WebRTC: PeerConnection created.
2021/07/24 07:12:11 Negotiating via BrokerChannel...
Target URL: snowflake-broker.torproject.net.global.prod.fastly.net
Front URL: cdn.sstatic.net
2021/07/24 07:12:11 WebRTC: closing DataChannel
2021/07/24 07:12:11 WebRTC: closing PeerConnection
2021/07/24 07:12:11 WebRTC: Closing
2021/07/24 07:12:11 WebRTC: dial tcp: lookup cdn.sstatic.net on [::1]:53: read udp [::1]:41939->[::1]:53: read: connection refused Retrying...
2021/07/24 07:12:21 WebRTC: Collecting a new Snowflake. Currently at [0/1]
2021/07/24 07:12:21 snowflake-07b5e298791ef573 connecting...
2021/07/24 07:12:21 WebRTC: DataChannel created.
2021/07/24 07:12:21 WebRTC: Created offer
2021/07/24 07:12:21 WebRTC: Set local description
2021/07/24 07:12:21 WebRTC: PeerConnection created.
2021/07/24 07:12:21 Negotiating via BrokerChannel...
Target URL: snowflake-broker.torproject.net.global.prod.fastly.net
Front URL: cdn.sstatic.net
2021/07/24 07:12:21 WebRTC: closing DataChannel
2021/07/24 07:12:21 WebRTC: closing PeerConnection
2021/07/24 07:12:21 WebRTC: Closing
2021/07/24 07:12:21 WebRTC: dial tcp: lookup cdn.sstatic.net on [::1]:53: read udp [::1]:40102->[::1]:53: read: connection refused Retrying...
```https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40046Occasional CI failure for Proxy tests2024-02-27T18:58:26ZCecylia BocovichOccasional CI failure for Proxy testsI've seen this CI failure pop up occasionally but not always which means it's probably a race condition. Not sure if the problem is in the tests or the proxy code proper.
```
=== RUN TestBrokerInteractions
Proxy connections to brok...I've seen this CI failure pop up occasionally but not always which means it's probably a race condition. Not sure if the problem is in the tests or the proxy code proper.
```
=== RUN TestBrokerInteractions
Proxy connections to broker
polls broker correctly ✔✔
handles poll error ✔2021/05/11 20:24:57 Error reading broker response: invalid character 'e' in literal true (expecting 'r')
2021/05/11 20:24:57 body: test
✔
sends answer to broker ✔✔✔✔
handles answer error ✔panic: test timed out after 10m0s
goroutine 10 [running]:
testing.(*M).startAlarm.func1()
/usr/local/go/src/testing/testing.go:1514 +0x11c
created by time.goFunc
/usr/local/go/src/time/sleep.go:168 +0x52
goroutine 1 [chan receive, 9 minutes]:
testing.(*T).Run(0xc0000e6c60, 0xd08040, 0x16, 0xd28e78, 0x1)
/usr/local/go/src/testing/testing.go:1096 +0x570
testing.runTests.func1(0xc0000e6c60)
/usr/local/go/src/testing/testing.go:1339 +0xa7
testing.tRunner(0xc0000e6c60, 0xc000133d50)
/usr/local/go/src/testing/testing.go:1050 +0x1ec
testing.runTests(0xc0000c7580, 0x124a120, 0x4, 0x4, 0x0)
/usr/local/go/src/testing/testing.go:1337 +0x595
testing.(*M).Run(0xc00010c200, 0x0)
/usr/local/go/src/testing/testing.go:1252 +0x300
main.main()
_testmain.go:50 +0x224
goroutine 23 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).run(0xc000436840, 0xdf7b00, 0xc000415b00, 0xc0000622a0, 0x0, 0x0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:176 +0x18a
github.com/pion/ice/v2.(*Agent).GetLocalCandidates(0xc000436840, 0x0, 0x0, 0x7, 0xc00002c313, 0x7)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:840 +0x12e
github.com/pion/webrtc/v3.(*ICEGatherer).GetLocalCandidates(0xc00042a630, 0xc0002acb88, 0x4788f7, 0x418040, 0xc0002acb80, 0x4787d7)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/icegatherer.go:216 +0x90
github.com/pion/webrtc/v3.populateLocalCandidates(0xc00058e380, 0xc00042a630, 0x2, 0x0)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/sdp.go:255 +0x88
github.com/pion/webrtc/v3.(*PeerConnection).CurrentLocalDescription(0xc0004581c0, 0x0)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:1928 +0x125
github.com/pion/webrtc/v3.(*PeerConnection).LocalDescription(0xc0004581c0, 0xc0002acdd8)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:993 +0x73
git.torproject.org/pluggable-transports/snowflake.git/proxy.(*SignalingServer).sendAnswer(0xc00047d2c0, 0xcffdba, 0x4, 0xc0004581c0, 0x0, 0x0)
/builds/cohosh/snowflake/proxy/snowflake.go:271 +0xe9
git.torproject.org/pluggable-transports/snowflake.git/proxy.TestBrokerInteractions.func1.4()
/builds/cohosh/snowflake/proxy/proxy-go_test.go:418 +0x270
github.com/smartystreets/goconvey/convey.parseAction.func1(0xdfc2c0, 0xc0000933e0)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/discovery.go:80 +0x3e
github.com/smartystreets/goconvey/convey.(*context).conveyInner(0xc0000933e0, 0xd0700b, 0x14, 0xc000582230)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/context.go:261 +0x21e
github.com/smartystreets/goconvey/convey.(*context).Convey.func1()
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/context.go:163 +0xa9
github.com/jtolds/gls.(*ContextManager).SetValues.func1(0x0)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/context.go:97 +0x5c5
github.com/jtolds/gls.EnsureGoroutineId(0xc000594300)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/gid.go:19 +0x1b4
github.com/jtolds/gls.(*ContextManager).SetValues(0xc000189750, 0xc0005942a0, 0xc00058e4a0)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/context.go:63 +0x26c
github.com/smartystreets/goconvey/convey.(*context).Convey(0xc000092600, 0xc0002ad688, 0x2, 0x2)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/context.go:162 +0x329
github.com/smartystreets/goconvey/convey.Convey(0xc0002ad688, 0x2, 0x2)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/doc.go:77 +0xb0
git.torproject.org/pluggable-transports/snowflake.git/proxy.TestBrokerInteractions.func1()
/builds/cohosh/snowflake/proxy/proxy-go_test.go:407 +0x728
github.com/smartystreets/goconvey/convey.parseAction.func1(0xdfc2c0, 0xc000092600)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/discovery.go:80 +0x3e
github.com/smartystreets/goconvey/convey.(*context).conveyInner(0xc000092600, 0xd0b399, 0x1b, 0xc0001d89c0)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/context.go:261 +0x21e
github.com/smartystreets/goconvey/convey.rootConvey.func1()
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/context.go:110 +0x1d1
github.com/jtolds/gls.(*ContextManager).SetValues.func1(0x0)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/context.go:97 +0x5c5
github.com/jtolds/gls.EnsureGoroutineId.func1()
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/gid.go:24 +0x46
github.com/jtolds/gls._m(0x0, 0xc0001cefa0)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/stack_tags.go:108 +0x43
github.com/jtolds/gls.github_com_jtolds_gls_markS(0x0, 0xc0001cefa0)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/stack_tags.go:56 +0x43
github.com/jtolds/gls.addStackTag(...)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/stack_tags.go:49
github.com/jtolds/gls.EnsureGoroutineId(0xc0001a7980)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/gid.go:24 +0x171
github.com/jtolds/gls.(*ContextManager).SetValues(0xc000189750, 0xc0001a7920, 0xc0001cef60)
/go/pkg/mod/github.com/jtolds/gls@v4.20.0+incompatible/context.go:63 +0x26c
github.com/smartystreets/goconvey/convey.rootConvey(0xc00005ae90, 0x3, 0x3)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/context.go:105 +0x458
github.com/smartystreets/goconvey/convey.Convey(0xc00005ae90, 0x3, 0x3)
/go/pkg/mod/github.com/smartystreets/goconvey@v1.6.4/convey/doc.go:75 +0xe1
git.torproject.org/pluggable-transports/snowflake.git/proxy.TestBrokerInteractions(0xc0001e8480)
/builds/cohosh/snowflake/proxy/proxy-go_test.go:339 +0xa7
testing.tRunner(0xc0001e8480, 0xd28e78)
/usr/local/go/src/testing/testing.go:1050 +0x1ec
created by testing.(*T).Run
/usr/local/go/src/testing/testing.go:1095 +0x538
goroutine 24 [select]:
github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).loop(0xc0000de550, 0xdecd40, 0xc0001d9710)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:126 +0x240
created by github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:65 +0x14b
goroutine 25 [select]:
github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).loop(0xc000192540, 0xdecd40, 0xc0001d9710)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:87 +0x277
created by github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:77 +0x14b
goroutine 26 [select]:
github.com/pion/interceptor/pkg/report.(*SenderInterceptor).loop(0xc0001925b0, 0xdecd40, 0xc0001d9710)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:97 +0x277
created by github.com/pion/interceptor/pkg/report.(*SenderInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:87 +0x14b
goroutine 27 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fc7672970f8, 0x72, 0xded640)
/usr/local/go/src/runtime/netpoll.go:203 +0x55
internal/poll.(*pollDesc).wait(0xc00010ce18, 0x72, 0xcc7600, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe4
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).RawRead(0xc00010ce00, 0xc0001fb230, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:534 +0x17e
net.(*rawConn).Read(0xc0000b46e0, 0xc0001fb230, 0x1, 0x1)
/usr/local/go/src/net/rawconn.go:43 +0x6f
golang.org/x/net/internal/socket.(*Conn).recvMsg(0xc0001ff420, 0xc00005ad70, 0x0, 0x0, 0x0)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/rawconn_msg.go:33 +0x3e2
golang.org/x/net/internal/socket.(*Conn).RecvMsg(...)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/socket.go:247
golang.org/x/net/ipv4.(*payloadHandler).ReadFrom(0xc0000df870, 0xc0001a4400, 0x200, 0x200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/ipv4/payload_cmsg.go:32 +0x211
github.com/pion/mdns.(*Conn).start(0xc00010d000)
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:256 +0x175
created by github.com/pion/mdns.Server
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:98 +0x714
goroutine 28 [select]:
github.com/pion/ice/v2.(*Agent).taskLoop(0xc0000f6dc0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:218 +0x24b
created by github.com/pion/ice/v2.NewAgent
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:349 +0x11f7
goroutine 29 [chan receive, 9 minutes]:
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func1(0xc0000f6dc0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:405 +0x83
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:401 +0x4d
goroutine 30 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func2(0xc0000f6dc0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:414 +0x15a
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:412 +0x6f
goroutine 35 [select, 9 minutes]:
context.propagateCancel.func1(0xdf7b00, 0xc00009cfc0, 0xdf0920, 0xc0002bc3c0)
/usr/local/go/src/context/context.go:276 +0x10f
created by context.propagateCancel
/usr/local/go/src/context/context.go:275 +0x271
goroutine 34 [select]:
github.com/pion/ice/v2.(*Agent).connectivityChecks(0xc0000f6dc0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:531 +0x364
created by github.com/pion/ice/v2.(*Agent).startConnectivityChecks.func1
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:472 +0x331
goroutine 33 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).connect(0xc0000f6dc0, 0xdf7380, 0xc0002bc0c0, 0xc0002bc000, 0xc0001ef02a, 0x4, 0xc0001fc3c8, 0x18, 0x0, 0x0, ...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:53 +0x231
github.com/pion/ice/v2.(*Agent).Accept(...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:21
github.com/pion/webrtc/v3.(*ICETransport).Start(0xc0001cca00, 0xc000180750, 0xc0001ef02a, 0x4, 0xc0001fc3c8, 0x18, 0x0, 0xc00005bea0, 0x0, 0x0)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/icetransport.go:137 +0x46e
github.com/pion/webrtc/v3.(*PeerConnection).startTransports(0xc0001f6000, 0x2, 0xc0ffffff01, 0xc0001ef02a, 0x4, 0xc0001fc3c8, 0x18, 0xc0000b1314, 0x5f, 0xc0000b130c, ...)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:2054 +0x169
github.com/pion/webrtc/v3.(*PeerConnection).SetRemoteDescription.func2()
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:1142 +0x17b
github.com/pion/webrtc/v3.(*operations).start(0xc0001a7c50)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:84 +0x7c
created by github.com/pion/webrtc/v3.(*operations).Enqueue
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:35 +0x19a
goroutine 4 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).run(0xc000436840, 0xdf7b00, 0xc000415b00, 0xc00000e380, 0xc00052a648, 0x4887d7)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:176 +0x18a
github.com/pion/ice/v2.(*Agent).connectivityChecks.func1()
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:481 +0x143
github.com/pion/ice/v2.(*Agent).connectivityChecks(0xc000436840)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:534 +0x41c
created by github.com/pion/ice/v2.(*Agent).startConnectivityChecks.func1
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:472 +0x331
goroutine 40 [select]:
github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).loop(0xc0000dff90, 0xdecd40, 0xc0002dabf0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:126 +0x240
created by github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:65 +0x14b
goroutine 41 [select]:
github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).loop(0xc000192f50, 0xdecd40, 0xc0002dabf0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:87 +0x277
created by github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:77 +0x14b
goroutine 42 [select]:
github.com/pion/interceptor/pkg/report.(*SenderInterceptor).loop(0xc000192fc0, 0xdecd40, 0xc0002dabf0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:97 +0x277
created by github.com/pion/interceptor/pkg/report.(*SenderInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:87 +0x14b
goroutine 43 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fc767296bb8, 0x72, 0xded640)
/usr/local/go/src/runtime/netpoll.go:203 +0x55
internal/poll.(*pollDesc).wait(0xc00010d898, 0x72, 0x7fc76685b300, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe4
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).RawRead(0xc00010d880, 0xc000412000, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:534 +0x17e
net.(*rawConn).Read(0xc0000b4790, 0xc000412000, 0x1, 0x1)
/usr/local/go/src/net/rawconn.go:43 +0x6f
golang.org/x/net/internal/socket.(*Conn).recvMsg(0xc0002f4260, 0xc00038fd70, 0x0, 0x0, 0x10)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/rawconn_msg.go:33 +0x3e2
golang.org/x/net/internal/socket.(*Conn).RecvMsg(...)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/socket.go:247
golang.org/x/net/ipv4.(*payloadHandler).ReadFrom(0xc0002dd1e0, 0xc000408000, 0x200, 0x200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/ipv4/payload_cmsg.go:32 +0x211
github.com/pion/mdns.(*Conn).start(0xc00010da80)
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:256 +0x175
created by github.com/pion/mdns.Server
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:98 +0x714
goroutine 44 [select]:
github.com/pion/ice/v2.(*Agent).taskLoop(0xc0000f7340)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:218 +0x24b
created by github.com/pion/ice/v2.NewAgent
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:349 +0x11f7
goroutine 45 [chan receive, 9 minutes]:
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func1(0xc0000f7340)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:405 +0x83
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:401 +0x4d
goroutine 46 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func2(0xc0000f7340)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:414 +0x15a
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:412 +0x6f
goroutine 82 [select]:
github.com/pion/ice/v2.(*Agent).connectivityChecks(0xc0000f7340)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:531 +0x364
created by github.com/pion/ice/v2.(*Agent).startConnectivityChecks.func1
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:472 +0x331
goroutine 49 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).connect(0xc0000f7340, 0xdf7380, 0xc00040a040, 0xc00040a000, 0xc0002d31fa, 0x4, 0xc0001fd8e8, 0x18, 0x0, 0x0, ...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:53 +0x231
github.com/pion/ice/v2.(*Agent).Accept(...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:21
github.com/pion/webrtc/v3.(*ICETransport).Start(0xc0001cd2c0, 0xc000180bd0, 0xc0002d31fa, 0x4, 0xc0001fd8e8, 0x18, 0x0, 0xc000390ea0, 0x0, 0x0)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/icetransport.go:137 +0x46e
github.com/pion/webrtc/v3.(*PeerConnection).startTransports(0xc0001f61c0, 0x2, 0xc0ffffff01, 0xc0002d31fa, 0x4, 0xc0001fd8e8, 0x18, 0xc0000b1d14, 0x5f, 0xc0000b1d0c, ...)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:2054 +0x169
github.com/pion/webrtc/v3.(*PeerConnection).SetRemoteDescription.func2()
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:1142 +0x17b
github.com/pion/webrtc/v3.(*operations).start(0xc0001fbbf0)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:84 +0x7c
created by github.com/pion/webrtc/v3.(*operations).Enqueue
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:35 +0x19a
goroutine 98 [select, 9 minutes]:
context.propagateCancel.func1(0xdf7b00, 0xc0002f6000, 0xdf0920, 0xc0002bd2c0)
/usr/local/go/src/context/context.go:276 +0x10f
created by context.propagateCancel
/usr/local/go/src/context/context.go:275 +0x271
goroutine 83 [select]:
github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).loop(0xc000404230, 0xdecd40, 0xc00040ceb0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:126 +0x240
created by github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:65 +0x14b
goroutine 84 [select]:
github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).loop(0xc00041e0e0, 0xdecd40, 0xc00040ceb0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:87 +0x277
created by github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:77 +0x14b
goroutine 85 [select]:
github.com/pion/interceptor/pkg/report.(*SenderInterceptor).loop(0xc00041e150, 0xdecd40, 0xc00040ceb0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:97 +0x277
created by github.com/pion/interceptor/pkg/report.(*SenderInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:87 +0x14b
goroutine 86 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fc767296e58, 0x72, 0xded640)
/usr/local/go/src/runtime/netpoll.go:203 +0x55
internal/poll.(*pollDesc).wait(0xc000420498, 0x72, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe4
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).RawRead(0xc000420480, 0xc0004139e0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:534 +0x17e
net.(*rawConn).Read(0xc000406048, 0xc0004139e0, 0x1, 0x1)
/usr/local/go/src/net/rawconn.go:43 +0x6f
golang.org/x/net/internal/socket.(*Conn).recvMsg(0xc00047c160, 0xc000391d70, 0x0, 0x0, 0x0)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/rawconn_msg.go:33 +0x3e2
golang.org/x/net/internal/socket.(*Conn).RecvMsg(...)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/socket.go:247
golang.org/x/net/ipv4.(*payloadHandler).ReadFrom(0xc000405460, 0xc000408600, 0x200, 0x200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/ipv4/payload_cmsg.go:32 +0x211
github.com/pion/mdns.(*Conn).start(0xc000420680)
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:256 +0x175
created by github.com/pion/mdns.Server
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:98 +0x714
goroutine 87 [select]:
github.com/pion/ice/v2.(*Agent).taskLoop(0xc0004362c0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:218 +0x24b
created by github.com/pion/ice/v2.NewAgent
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:349 +0x11f7
goroutine 88 [chan receive, 9 minutes]:
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func1(0xc0004362c0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:405 +0x83
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:401 +0x4d
goroutine 89 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func2(0xc0004362c0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:414 +0x15a
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:412 +0x6f
goroutine 93 [select, 9 minutes]:
context.propagateCancel.func1(0xdf7b00, 0xc000414a20, 0xdf0920, 0xc00040b000)
/usr/local/go/src/context/context.go:276 +0x10f
created by context.propagateCancel
/usr/local/go/src/context/context.go:275 +0x271
goroutine 92 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).connect(0xc0004362c0, 0xdf7380, 0xc00040ad80, 0xc00040ad00, 0xc00041125a, 0x4, 0xc00040ee88, 0x18, 0x0, 0x0, ...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:53 +0x231
github.com/pion/ice/v2.(*Agent).Accept(...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:21
github.com/pion/webrtc/v3.(*ICETransport).Start(0xc0004600a0, 0xc00042a1b0, 0xc00041125a, 0x4, 0xc00040ee88, 0x18, 0x0, 0xc000512ea0, 0x0, 0x0)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/icetransport.go:137 +0x46e
github.com/pion/webrtc/v3.(*PeerConnection).startTransports(0xc000458000, 0x2, 0xc0ffffff01, 0xc00041125a, 0x4, 0xc00040ee88, 0x18, 0xc000416694, 0x5f, 0xc00041668c, ...)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:2054 +0x169
github.com/pion/webrtc/v3.(*PeerConnection).SetRemoteDescription.func2()
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:1142 +0x17b
github.com/pion/webrtc/v3.(*operations).start(0xc000412480)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:84 +0x7c
created by github.com/pion/webrtc/v3.(*operations).Enqueue
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:35 +0x19a
goroutine 95 [select]:
github.com/pion/ice/v2.(*Agent).connectivityChecks(0xc0004362c0)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:531 +0x364
created by github.com/pion/ice/v2.(*Agent).startConnectivityChecks.func1
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:472 +0x331
goroutine 117 [select]:
github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).loop(0xc000405b80, 0xdecd40, 0xc0005465d0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:126 +0x240
created by github.com/pion/interceptor/pkg/nack.(*GeneratorInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/nack/generator_interceptor.go:65 +0x14b
goroutine 118 [select]:
github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).loop(0xc00041ebd0, 0xdecd40, 0xc0005465d0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:87 +0x277
created by github.com/pion/interceptor/pkg/report.(*ReceiverInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/receiver_interceptor.go:77 +0x14b
goroutine 119 [select]:
github.com/pion/interceptor/pkg/report.(*SenderInterceptor).loop(0xc00041ec40, 0xdecd40, 0xc0005465d0)
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:97 +0x277
created by github.com/pion/interceptor/pkg/report.(*SenderInterceptor).BindRTCPWriter
/go/pkg/mod/github.com/pion/interceptor@v0.0.10/pkg/report/sender_interceptor.go:87 +0x14b
goroutine 120 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fc767296678, 0x72, 0xded640)
/usr/local/go/src/runtime/netpoll.go:203 +0x55
internal/poll.(*pollDesc).wait(0xc000421418, 0x72, 0x7fc7666c3b00, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe4
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).RawRead(0xc000421400, 0xc000594000, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:534 +0x17e
net.(*rawConn).Read(0xc000406100, 0xc000594000, 0x1, 0x1)
/usr/local/go/src/net/rawconn.go:43 +0x6f
golang.org/x/net/internal/socket.(*Conn).recvMsg(0xc0005491e0, 0xc00058ad70, 0x0, 0x0, 0x10)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/rawconn_msg.go:33 +0x3e2
golang.org/x/net/internal/socket.(*Conn).RecvMsg(...)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/internal/socket/socket.go:247
golang.org/x/net/ipv4.(*payloadHandler).ReadFrom(0xc000544e20, 0xc000584000, 0x200, 0x200, 0x0, 0x0, 0x0, 0x0, 0x0, 0x0)
/go/pkg/mod/golang.org/x/net@v0.0.0-20210316092652-d523dce5a7f4/ipv4/payload_cmsg.go:32 +0x211
github.com/pion/mdns.(*Conn).start(0xc000421600)
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:256 +0x175
created by github.com/pion/mdns.Server
/go/pkg/mod/github.com/pion/mdns@v0.0.4/conn.go:98 +0x714
goroutine 121 [chan send, 9 minutes]:
github.com/pion/ice/v2.(*Agent).addCandidate.func1(0xdf7b00, 0xc000415b00, 0xc000436840)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:832 +0x658
github.com/pion/ice/v2.(*Agent).taskLoop(0xc000436840)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:222 +0x116
created by github.com/pion/ice/v2.NewAgent
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:349 +0x11f7
goroutine 122 [chan receive, 9 minutes]:
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func1(0xc000436840)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:405 +0x83
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:401 +0x4d
goroutine 123 [semacquire, 9 minutes]:
sync.runtime_SemacquireMutex(0xc0004581d4, 0x900000000, 0x1)
/usr/local/go/src/runtime/sema.go:71 +0x47
sync.(*Mutex).lockSlow(0xc0004581d0)
/usr/local/go/src/sync/mutex.go:138 +0x1c1
sync.(*Mutex).Lock(0xc0004581d0)
/usr/local/go/src/sync/mutex.go:81 +0x7d
sync.(*RWMutex).Lock(0xc0004581d0)
/usr/local/go/src/sync/rwmutex.go:98 +0x4a
github.com/pion/webrtc/v3.(*PeerConnection).onICEConnectionStateChange(0xc0004581c0, 0x2)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:467 +0x48
github.com/pion/webrtc/v3.(*PeerConnection).createICETransport.func1(0x2)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:777 +0x6f
github.com/pion/webrtc/v3.(*ICETransport).onConnectionStateChange(0xc000460aa0, 0x2)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/icetransport.go:221 +0x6c
github.com/pion/webrtc/v3.(*ICETransport).Start.func1(0x2)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/icetransport.go:101 +0xad
github.com/pion/ice/v2.(*Agent).onConnectionStateChange(0xc000436840, 0x2)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:396 +0x77
github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine.func2(0xc000436840)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:422 +0x18c
created by github.com/pion/ice/v2.(*Agent).startOnConnectionStateChangeRoutine
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:412 +0x6f
goroutine 132 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).connect(0xc000436840, 0xdf7380, 0xc000020180, 0xc000020100, 0xc00051762a, 0x4, 0xc0005425e8, 0x18, 0x0, 0x0, ...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:53 +0x231
github.com/pion/ice/v2.(*Agent).Accept(...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/transport.go:21
github.com/pion/webrtc/v3.(*ICETransport).Start(0xc000460aa0, 0xc00042a630, 0xc00051762a, 0x4, 0xc0005425e8, 0x18, 0x0, 0xc000060ea0, 0x0, 0x0)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/icetransport.go:137 +0x46e
github.com/pion/webrtc/v3.(*PeerConnection).startTransports(0xc0004581c0, 0x2, 0xc0ffffff01, 0xc00051762a, 0x4, 0xc0005425e8, 0x18, 0xc000417594, 0x5f, 0xc00041758c, ...)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:2054 +0x169
github.com/pion/webrtc/v3.(*PeerConnection).SetRemoteDescription.func2()
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/peerconnection.go:1142 +0x17b
github.com/pion/webrtc/v3.(*operations).start(0xc00051a540)
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:84 +0x7c
created by github.com/pion/webrtc/v3.(*operations).Enqueue
/go/pkg/mod/github.com/pion/webrtc/v3@v3.0.15/operations.go:35 +0x19a
goroutine 133 [select, 9 minutes]:
context.propagateCancel.func1(0xdf7b00, 0xc000415b00, 0xdf0920, 0xc000590280)
/usr/local/go/src/context/context.go:276 +0x10f
created by context.propagateCancel
/usr/local/go/src/context/context.go:275 +0x271
goroutine 134 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc000580488)
/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc000580480)
/usr/local/go/src/sync/waitgroup.go:130 +0xd4
github.com/pion/ice/v2.(*Agent).gatherCandidates(0xc000436840, 0xdf7380, 0xc000590280)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:120 +0x3ca
created by github.com/pion/ice/v2.(*Agent).GatherCandidates.func1
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:75 +0x1b3
goroutine 135 [chan receive, 9 minutes]:
github.com/pion/ice/v2.(*Agent).run(0xc000436840, 0xdf7380, 0xc000590280, 0xc0005943f0, 0x70dfad, 0xc0005b8180)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:180 +0x1b5
github.com/pion/ice/v2.(*Agent).addCandidate(0xc000436840, 0xdf7380, 0xc000590280, 0xdff720, 0xc000598b60, 0xdfaa60, 0xc00059e020, 0xc000594360, 0xdfcc20)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:808 +0x11c
github.com/pion/ice/v2.(*Agent).gatherCandidatesLocal(0xc000436840, 0xdf7380, 0xc000590280, 0xc000517950, 0x2, 0x2)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:210 +0x64f
github.com/pion/ice/v2.(*Agent).gatherCandidates.func1(0xc000436840, 0xdf7380, 0xc000590280, 0xc000580480)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:94 +0x8c
created by github.com/pion/ice/v2.(*Agent).gatherCandidates
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:93 +0x1a0
goroutine 136 [semacquire, 9 minutes]:
sync.runtime_Semacquire(0xc000684018)
/usr/local/go/src/runtime/sema.go:56 +0x42
sync.(*WaitGroup).Wait(0xc000684010)
/usr/local/go/src/sync/waitgroup.go:130 +0xd4
github.com/pion/ice/v2.(*Agent).gatherCandidatesSrflx(0xc000436840, 0xdf7380, 0xc000590280, 0xc0004060d8, 0x1, 0x1, 0xc000517950, 0x2, 0x2)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:333 +0x2ea
github.com/pion/ice/v2.(*Agent).gatherCandidates.func2(0xc000436840, 0xdf7380, 0xc000590280, 0xc000580480)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:100 +0xe6
created by github.com/pion/ice/v2.(*Agent).gatherCandidates
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:99 +0x2ab
goroutine 138 [IO wait, 9 minutes]:
internal/poll.runtime_pollWait(0x7fc767296758, 0x72, 0x0)
/usr/local/go/src/runtime/netpoll.go:203 +0x55
internal/poll.(*pollDesc).wait(0xc00059c218, 0x72, 0x2000, 0x2000, 0x0)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:87 +0xe4
internal/poll.(*pollDesc).waitRead(...)
/usr/local/go/src/internal/poll/fd_poll_runtime.go:92
internal/poll.(*FD).ReadFrom(0xc00059c200, 0xc0005c6000, 0x2000, 0x2000, 0x0, 0x0, 0x0, 0x0, 0x0)
/usr/local/go/src/internal/poll/fd_unix.go:219 +0x228
net.(*netFD).readFrom(0xc00059c200, 0xc0005c6000, 0x2000, 0x2000, 0xc000526da8, 0xcea4e30c8d38f, 0x0, 0x0, 0x2)
/usr/local/go/src/net/fd_unix.go:208 +0x7a
net.(*UDPConn).readFrom(0xc00059e020, 0xc0005c6000, 0x2000, 0x2000, 0xc000415b60, 0x7fc76741c2f0, 0x0, 0xc0005c6000)
/usr/local/go/src/net/udpsock_posix.go:47 +0x8e
net.(*UDPConn).ReadFrom(0xc00059e020, 0xc0005c6000, 0x2000, 0x2000, 0xc000526e00, 0x4b5ab5, 0x4ba056, 0xc000526f70, 0x4ba060)
/usr/local/go/src/net/udpsock.go:118 +0x95
github.com/pion/ice/v2.(*candidateBase).recvLoop(0xc000598b60, 0xc000415b60)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/candidate_base.go:222 +0x2d5
created by github.com/pion/ice/v2.(*candidateBase).start
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/candidate_base.go:205 +0x19f
goroutine 146 [select, 9 minutes]:
github.com/pion/ice/v2.(*Agent).run(0xc000436840, 0xdf7380, 0xc000590280, 0xc0000ab6e0, 0xc000209c60, 0x43c995)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:176 +0x18a
github.com/pion/ice/v2.(*Agent).addCandidate(0xc000436840, 0xdf7380, 0xc000590280, 0xdff9c0, 0xc0002760c0, 0xdfaa60, 0xc000010020, 0x0, 0xdfcc20)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/agent.go:808 +0x11c
github.com/pion/ice/v2.(*Agent).gatherCandidatesSrflx.func1(0xc000684010, 0xc000436840, 0xdf7380, 0xc000590280, 0x1, 0xd0c6f3, 0x11, 0x4b66, 0x0, 0x0, ...)
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:324 +0xf0e
created by github.com/pion/ice/v2.(*Agent).gatherCandidatesSrflx
/go/pkg/mod/github.com/pion/ice/v2@v2.0.15/gather.go:285 +0x2af
FAIL git.torproject.org/pluggable-transports/snowflake.git/proxy 600.055s
? git.torproject.org/pluggable-transports/snowflake.git/server [no test files]
```https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40045IPtProxy/Onion Browser: Snowflake client just stops with errors when bootstra...2021-05-26T12:34:08ZtlaIPtProxy/Onion Browser: Snowflake client just stops with errors when bootstrapping Tor on iOS (and therefore the whole bootstrap)We're experiencing a lot of instability with Snowflake on Android and iOS.
Esp. on iOS, I always seem to get only the "unrestricted" types. What exactly are the preconditions, to get to the "restricted" snowflakes? Is there something we...We're experiencing a lot of instability with Snowflake on Android and iOS.
Esp. on iOS, I always seem to get only the "unrestricted" types. What exactly are the preconditions, to get to the "restricted" snowflakes? Is there something we can do?
Latest IPtProxy currently uses Snowflake at this commit: https://gitweb.torproject.org/pluggable-transports/snowflake.git/commit/?id=af6e2c30e1a6aacc6e7adf9a31df0a387891cc37
While trying to understand, why Snowflake connections are often delayed a long time or not happening at all, I experienced random stops of the code. This is esp. on starting up Onion Browser on iOS.
E.g. see the 2 different logs below.
(It's Onion Browser, Tor and Snowflake combined.)
That's all. That's not just a part. After the last log line, everything stopped.
(If the user wouldn't change bridge settings, that is...)
The change is about 50/50 that it just stops.
Do you have an idea, why that happens?
Interestingly enough, this seems to happen on a real device way more often than on a simulator on an Intel MacBook Pro running Big Sur.
(Note simulator vs. emulator!)
So that might point to a timing issue?
This is our config:
https://github.com/OnionBrowser/OnionBrowser/blob/2.X/OnionBrowser/OnionManager.swift#L245-L248
Is there something we can do to mitigate this in IPtProxy? Some catch-exception-and-restart or something?
```
2021-05-11 17:04:17.463104+0200 OnionBrowser[4397:1863717] [HSTSCache] locked and loaded with 67296 preloaded hosts
[OnionManager] dataDir=file:///var/mobile/Containers/Data/Application/164FD3FE-402A-4A05-A626-1F4F48B6573C/Library/Caches/tor/
[OnionManager] bridgesId=snowflake
[OnionManager] #stopObfs4proxy
2021/05/11 15:04:18
--- Starting Snowflake Client ---
2021/05/11 15:04:18 Using ICE servers:
2021/05/11 15:04:18 url: stun:stun.voipgate.com:3478
2021/05/11 15:04:18 url: stun:stun.uls.co.za:3478
2021/05/11 15:04:18 url: stun:stun.altar.com.pl:3478
2021/05/11 15:04:18 url: stun:stun.voys.nl:3478
2021/05/11 15:04:18 url: stun:stun.dus.net:3478
2021/05/11 15:04:18 url: stun:stun.epygi.com:3478
2021/05/11 15:04:18 url: stun:stun.sonetel.com:3478
2021/05/11 15:04:18 Rendezvous using Broker at: https://snowflake-broker.torproject.net.global.prod.fastly.net/
2021/05/11 15:04:18 Domain fronting using: cdn.sstatic.net
VERSION 1
CMETHOD-ERROR meek_lite no such method
CMETHOD-ERROR obfs2 no such method
CMETHOD-ERROR obfs3 no such method
CMETHOD-ERROR obfs4 no such method
CMETHOD-ERROR scramblesuit no such method
[OnionManager] ipv6_status: 0
[OnionManager] arguments=["--allow-missing-torrc", "--ignore-missing-torrc", "--ClientOnly", "1", "--AvoidDiskWrites", "1", "--SocksPort", "127.0.0.1:39050", "--ControlPort", "127.0.0.1:39060", "--Log", "notice stdout", "--ClientUseIPv6", "1", "--ClientTransportPlugin", "obfs4 socks5 127.0.0.1:47351", "--ClientTransportPlugin", "meek_lite socks5 127.0.0.1:47352", "--ClientTransportPlugin", "snowflake socks5 127.0.0.1:52610", "--GeoIPFile", "/private/var/containers/Bundle/Application/B0BCF18E-0C46-41A6-A093-4B4879EB264C/OnionBrowser.app/geoip", "--GeoIPv6File", "/private/var/containers/Bundle/Application/B0BCF18E-0C46-41A6-A093-4B4879EB264C/OnionBrowser.app/geoip6", "--ClientOnionAuthDir", "/var/mobile/Containers/Data/Application/164FD3FE-402A-4A05-A626-1F4F48B6573C/Library/Caches/tor/auth", "--Bridge", "snowflake 192.0.2.3:1", "--UseBridges", "1", "--ClientPreferIPv6ORPort", "auto", "--ClientUseIPv4", "1"]
[OnionManager] Starting Tor
2021/05/11 15:04:18 Started SOCKS listener at 127.0.0.1:52610.
CMETHOD snowflake socks5 127.0.0.1:52610
CMETHODS DONE
2021-05-11 17:04:18.232173+0200 OnionBrowser[4397:1863717] [Assert] Trying to present the presentation controller while transitioning already. (<_UIRootPresentationController: 0x111d17d30>)
May 11 17:04:18.251 [notice] Tor 0.4.5.7 (git-83f895c015de5520) running on Darwin with Libevent 2.1.12-stable, OpenSSL 1.1.1k, Zlib 1.2.11, Liblzma 5.2.5, Libzstd N/A and Unknown N/A as libc.
May 11 17:04:18.251 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
May 11 17:04:18.251 [notice] Configuration file "/private/var/mobile/Containers/Data/Application/164FD3FE-402A-4A05-A626-1F4F48B6573C/.torrc" not present, using reasonable defaults.
May 11 17:04:18.256 [notice] Opening Socks listener on 127.0.0.1:39050
May 11 17:04:18.256 [notice] Opened Socks listener connection (ready) on 127.0.0.1:39050
May 11 17:04:18.256 [notice] Opening Control listener on 127.0.0.1:39060
May 11 17:04:18.256 [notice] Opened Control listener connection (ready) on 127.0.0.1:39060
May 11 17:04:18.000 [notice] Parsing GEOIP IPv4 file /private/var/containers/Bundle/Application/B0BCF18E-0C46-41A6-A093-4B4879EB264C/OnionBrowser.app/geoip.
2021/05/11 15:04:18 NAT Type: unrestricted
May 11 17:04:18.000 [notice] Parsing GEOIP IPv6 file /private/var/containers/Bundle/Application/B0BCF18E-0C46-41A6-A093-4B4879EB264C/OnionBrowser.app/geoip6.
May 11 17:04:18.000 [notice] Bootstrapped 0% (starting): Starting
May 11 17:04:18.000 [notice] Starting with guard context "bridges"
May 11 17:04:18.000 [notice] Delaying directory fetches: No running bridges
May 11 17:04:19.000 [notice] New control connection opened from 127.0.0.1.
[OnionManager] cookie= XEkOaClqVqXuS7OS0W9SkYS8Qpf9TxnSdRNrqRZngEA=
May 11 17:04:19.000 [notice] Bootstrapped 1% (conn_pt): Connecting to pluggable transport
May 11 17:04:19.000 [notice] Bootstrapped 2% (conn_done_pt): Connected to pluggable transport
2021/05/11 15:04:19 SOCKS accepted: {192.0.2.3:1 map[]}
2021/05/11 15:04:19 ---- Handler: begin collecting snowflakes ---
2021/05/11 15:04:19 ---- Handler: starting a new session ---
May 11 17:04:19.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
2021/05/11 15:04:19 WebRTC: Collecting a new Snowflake. Currently at [0/3]
2021/05/11 15:04:19 snowflake-b5939212541b4c43 connecting...
2021/05/11 15:04:19 redialing on same connection
[OnionManager] progress=1
[OnionManager] progress=2
2021/05/11 15:04:19 ---- Handler: begin stream 3 ---
[OnionManager] progress=10
2021/05/11 15:04:20 WebRTC: DataChannel created.
2021/05/11 15:04:20 WebRTC: Created offer
2021/05/11 15:04:20 WebRTC: Set local description
2021/05/11 15:04:20 WebRTC: PeerConnection created.
2021/05/11 15:04:20 Negotiating via BrokerChannel...
Target URL: snowflake-broker.torproject.net.global.prod.fastly.net Front URL: cdn.sstatic.net
2021/05/11 15:04:23 BrokerChannel Response:
200 OK
2021/05/11 15:04:23 Received answer: {"type":"answer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-87.0 5216436289655477355 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=sendrecv\r\na=fingerprint:sha-256 E9:DE:10:5F:A8:C9:7F:D1:49:1F:26:49:DF:56:7C:7F:97:C5:DE:DD:EF:D7:E6:6D:DD:E3:66:3F:D2:6F:31:49\r\na=group:BUNDLE 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application 46050 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 85.114.140.58\r\na=candidate:0 1 UDP 2122187007 65ea5e9f-adaf-4680-989d-c6b2f66a0d14.local 46050 typ host\r\na=candidate:2 1 UDP 2122252543 5d3245fa-bea1-4e18-ab51-954ee57e291f.local 59991 typ host\r\na=candidate:4 1 TCP 2105458943 65ea5e9f-adaf-4680-989d-c6b2f66a0d14.local 9 typ host tcptype active\r\na=candidate:5 1 TCP 2105524479 5d3245fa-bea1-4e18-ab51-954ee57e291f.local 9 typ host tcptype active\r\na=candidate:1 1 UDP 1685987327 85.114.140.58 46050 typ srflx raddr 0.0.0.0 rport 0\r\na=candidate:3 1 UDP 1686052607 2a01:4f8:191:22f6:77bf:d860:9b03:d768 59991 typ srflx raddr 0.0.0.0 rport 0\r\na=sendrecv\r\na=end-of-candidates\r\na=ice-pwd:b2a69faea3356249fb9abf124a9dfec3\r\na=ice-ufrag:e5f0ed04\r\na=mid:0\r\na=setup:active\r\na=sctp-port:5000\r\na=max-message-size:1073741823\r\n"}
2021/05/11 15:04:23 Received Answer.
2021/05/11 15:04:25 WebRTC: DataChannel.OnOpen
2021/05/11 15:04:25 ---- Handler: snowflake assigned ----
May 11 17:04:26.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
[OnionManager] progress=14
May 11 17:04:26.000 [notice] Learned fingerprint 2B280B23E1107BB62ABFC40DDCC8824814F80A72 for bridge 192.0.2.3:1 (with transport 'snowflake').
May 11 17:04:26.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
May 11 17:04:26.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
[OnionManager] progress=15
[OnionManager] progress=20
May 11 17:04:27.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
[OnionManager] progress=25
May 11 17:04:27.000 [notice] new bridge descriptor 'flakey' (fresh): $2B280B23E1107BB62ABFC40DDCC8824814F80A72~flakey [1zOHpg+FxqQfi/6jDLtCpHHqBTH8gjYmCKXkus1D5Ko] at 192.0.2.3
2021/05/11 15:04:29 Traffic Bytes (in|out): 9355 | 7290 -- (19 OnMessages, 22 Sends)
2021/05/11 15:04:29 WebRTC: Collecting a new Snowflake. Currently at [1/3]
2021/05/11 15:04:29 snowflake-69370291c40e5fbb connecting...
2021/05/11 15:04:29 WebRTC: DataChannel created.
2021/05/11 15:04:29 WebRTC: Created offer
2021/05/11 15:04:29 WebRTC: Set local description
2021/05/11 15:04:29 WebRTC: PeerConnection created.
2021/05/11 15:04:29 Negotiating via BrokerChannel...
Target URL: snowflake-broker.torproject.net.global.prod.fastly.net Front URL: cdn.sstatic.net
[OnionManager] Triggering Tor connection retry.
May 11 17:04:33.000 [notice] Closing no-longer-configured Socks listener on 127.0.0.1:39050
May 11 17:04:33.000 [notice] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections.
May 11 17:04:33.000 [notice] Opening Socks listener on 127.0.0.1:39050
May 11 17:04:33.000 [notice] Opened Socks listener connection (ready) on 127.0.0.1:39050
2021/05/11 15:04:33 copy loop ended
2021/05/11 15:04:33 ---- Handler: closed stream 3 ---
2021/05/11 15:04:34 BrokerChannel Response:
200 OK
2021/05/11 15:04:34 Received answer: {"type":"answer","sdp":"v=0\r\no=- 7009743295814005219 2 IN IP4 127.0.0.1\r\ns=-\r\nt=0 0\r\na=group:BUNDLE 0\r\na=msid-semantic: WMS\r\nm=application 42411 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 85.107.124.19\r\na=candidate:111704799 1 udp 2122260223 192.168.1.34 54705 typ host generation 0 network-id 1 network-cost 10\r\na=candidate:1210811951 1 tcp 1518280447 192.168.1.34 9 typ host tcptype active generation 0 network-id 1 network-cost 10\r\na=candidate:4038591243 1 udp 1686052607 85.107.124.19 42411 typ srflx raddr 192.168.1.34 rport 54705 generation 0 network-id 1 network-cost 10\r\na=ice-ufrag:t5qZ\r\na=ice-pwd:AiKgb8nIocPjprb7/w469Q28\r\na=ice-options:trickle\r\na=fingerprint:sha-256 0E:C5:3A:6E:E7:87:9A:9A:08:13:25:C0:24:2E:92:E7:A6:87:5E:A2:53:9A:AA:3F:96:23:8F:E2:05:26:FE:5D\r\na=setup:active\r\na=mid:0\r\na=sctp-port:5000\r\n"}
2021/05/11 15:04:34 Received Answer.
2021/05/11 15:04:34 Traffic Bytes (in|out): 7168 | 1323 -- (10 OnMessages, 9 Sends)
2021/05/11 15:04:36 WebRTC: DataChannel.OnOpen
2021/05/11 15:04:36 ConnectLoop: stopped.
2021/05/11 15:04:36 WebRTC: closing DataChannel
2021/05/11 15:04:36 WebRTC: closing PeerConnection
2021/05/11 15:04:36 WebRTC: DataChannel.OnClose
2021/05/11 15:04:36 WebRTC: Closing
2021/05/11 15:04:36 WebRTC: closing DataChannel
2021/05/11 15:04:36 redialing on same connection
2021/05/11 15:04:36 WebRTC: closing PeerConnection
2021/05/11 15:04:36 WebRTC: DataChannel.OnClose
2021/05/11 15:04:36 copying WebRTC to SOCKS resulted in error: read dummy dummy->dummy: handler: Received invalid Snowflake
2021/05/11 15:04:36 WebRTC: Closing
2021/05/11 15:04:36 WebRTC: melted all 2 snowflakes.
2021/05/11 15:04:36 ---- Handler: end collecting snowflakes ---
2021/05/11 15:04:36 ---- Handler: discarding finished session ---
2021/05/11 15:04:36 Handler ended
```
```
2021-05-11 17:23:07.954229+0200 OnionBrowser[4416:1871406] [HSTSCache] locked and loaded with 67296 preloaded hosts
[OnionManager] dataDir=file:///var/mobile/Containers/Data/Application/173396C5-9677-4A89-9DBE-A0275AF0102A/Library/Caches/tor/
[OnionManager] bridgesId=snowflake
[OnionManager] #stopObfs4proxy
2021/05/11 15:23:08
--- Starting Snowflake Client ---
2021/05/11 15:23:08 Using ICE servers:
2021/05/11 15:23:08 url: stun:stun.altar.com.pl:3478
2021/05/11 15:23:08 url: stun:stun.antisip.com:3478
2021/05/11 15:23:08 url: stun:stun.voys.nl:3478
2021/05/11 15:23:08 url: stun:stun.voip.blackberry.com:3478
2021/05/11 15:23:08 url: stun:stun.uls.co.za:3478
2021/05/11 15:23:08 url: stun:stun.bluesip.net:3478
2021/05/11 15:23:08 url: stun:stun.voipgate.com:3478
2021/05/11 15:23:08 Rendezvous using Broker at: https://snowflake-broker.torproject.net.global.prod.fastly.net/
2021/05/11 15:23:08 Domain fronting using: cdn.sstatic.net
[OnionManager] ipv6_status: 0
VERSION 1
CMETHOD-ERROR meek_lite no such method
CMETHOD-ERROR obfs2 no such method
CMETHOD-ERROR obfs3 no such method
CMETHOD-ERROR obfs4 no such method
CMETHOD-ERROR scramblesuit no such method
[OnionManager] arguments=["--allow-missing-torrc", "--ignore-missing-torrc", "--ClientOnly", "1", "--AvoidDiskWrites", "1", "--SocksPort", "127.0.0.1:39050", "--ControlPort", "127.0.0.1:39060", "--Log", "notice stdout", "--ClientUseIPv6", "1", "--ClientTransportPlugin", "obfs4 socks5 127.0.0.1:47351", "--ClientTransportPlugin", "meek_lite socks5 127.0.0.1:47352", "--ClientTransportPlugin", "snowflake socks5 127.0.0.1:52610", "--GeoIPFile", "/private/var/containers/Bundle/Application/D62B6AF8-18A6-4C05-95D3-0A1C0AD669DE/OnionBrowser.app/geoip", "--GeoIPv6File", "/private/var/containers/Bundle/Application/D62B6AF8-18A6-4C05-95D3-0A1C0AD669DE/OnionBrowser.app/geoip6", "--ClientOnionAuthDir", "/var/mobile/Containers/Data/Application/173396C5-9677-4A89-9DBE-A0275AF0102A/Library/Caches/tor/auth", "--Bridge", "snowflake 192.0.2.3:1", "--UseBridges", "1", "--ClientPreferIPv6ORPort", "auto", "--ClientUseIPv4", "1"]
2021/05/11 15:23:08 Started SOCKS listener at 127.0.0.1:52610.
CMETHOD snowflake socks5 127.0.0.1:52610
[OnionManager] Starting Tor
CMETHODS DONE
2021-05-11 17:23:08.707322+0200 OnionBrowser[4416:1871406] [Assert] Trying to present the presentation controller while transitioning already. (<_UIRootPresentationController: 0x10850ce10>)
May 11 17:23:08.722 [notice] Tor 0.4.5.7 (git-83f895c015de5520) running on Darwin with Libevent 2.1.12-stable, OpenSSL 1.1.1k, Zlib 1.2.11, Liblzma 5.2.5, Libzstd N/A and Unknown N/A as libc.
May 11 17:23:08.722 [notice] Tor can't help you if you use it wrong! Learn how to be safe at https://www.torproject.org/download/download#warning
May 11 17:23:08.723 [notice] Configuration file "/private/var/mobile/Containers/Data/Application/173396C5-9677-4A89-9DBE-A0275AF0102A/.torrc" not present, using reasonable defaults.
May 11 17:23:08.726 [notice] Opening Socks listener on 127.0.0.1:39050
May 11 17:23:08.726 [notice] Opened Socks listener connection (ready) on 127.0.0.1:39050
May 11 17:23:08.726 [notice] Opening Control listener on 127.0.0.1:39060
May 11 17:23:08.726 [notice] Opened Control listener connection (ready) on 127.0.0.1:39060
May 11 17:23:08.000 [notice] Parsing GEOIP IPv4 file /private/var/containers/Bundle/Application/D62B6AF8-18A6-4C05-95D3-0A1C0AD669DE/OnionBrowser.app/geoip.
2021/05/11 15:23:08 NAT Type: unrestricted
May 11 17:23:08.000 [notice] Parsing GEOIP IPv6 file /private/var/containers/Bundle/Application/D62B6AF8-18A6-4C05-95D3-0A1C0AD669DE/OnionBrowser.app/geoip6.
May 11 17:23:08.000 [notice] Bootstrapped 0% (starting): Starting
May 11 17:23:09.000 [notice] Starting with guard context "bridges"
May 11 17:23:09.000 [notice] Delaying directory fetches: No running bridges
May 11 17:23:09.000 [notice] New control connection opened from 127.0.0.1.
[OnionManager] cookie= tjBUpSa7mqXyVlHo25Xszzu170VQ5Tm2HBfzepFXwqk=
May 11 17:23:10.000 [notice] Bootstrapped 1% (conn_pt): Connecting to pluggable transport
May 11 17:23:10.000 [notice] Bootstrapped 2% (conn_done_pt): Connected to pluggable transport
2021/05/11 15:23:10 SOCKS accepted: {192.0.2.3:1 map[]}
2021/05/11 15:23:10 ---- Handler: begin collecting snowflakes ---
2021/05/11 15:23:10 ---- Handler: starting a new session ---
2021/05/11 15:23:10 WebRTC: Collecting a new Snowflake. Currently at [0/3]
2021/05/11 15:23:10 snowflake-cdad34980557846d connecting...
May 11 17:23:10.000 [notice] Bootstrapped 10% (conn_done): Connected to a relay
2021/05/11 15:23:10 redialing on same connection
2021/05/11 15:23:10 ---- Handler: begin stream 3 ---
2021/05/11 15:23:10 WebRTC: DataChannel created.
2021/05/11 15:23:10 WebRTC: Created offer
2021/05/11 15:23:10 WebRTC: Set local description
2021/05/11 15:23:10 WebRTC: PeerConnection created.
2021/05/11 15:23:10 Negotiating via BrokerChannel...
Target URL: snowflake-broker.torproject.net.global.prod.fastly.net Front URL: cdn.sstatic.net
2021/05/11 15:23:12 BrokerChannel Response:
200 OK
2021/05/11 15:23:12 Received answer: {"type":"answer","sdp":"v=0\r\no=mozilla...THIS_IS_SDPARTA-89.0 3176598337624981171 0 IN IP4 0.0.0.0\r\ns=-\r\nt=0 0\r\na=sendrecv\r\na=fingerprint:sha-256 6B:41:FC:99:15:80:AE:56:9A:B1:31:6B:9A:D1:1D:A4:91:13:71:1D:11:BD:78:15:C8:D9:31:33:03:AE:71:98\r\na=group:BUNDLE 0\r\na=ice-options:trickle\r\na=msid-semantic:WMS *\r\nm=application 61455 UDP/DTLS/SCTP webrtc-datachannel\r\nc=IN IP4 67.171.207.78\r\na=candidate:0 1 UDP 2122187007 65d476bf-60a6-4ec2-b9bb-99ca00f2c4ac.local 61455 typ host\r\na=candidate:2 1 UDP 2122252543 67213273-8df2-472b-82c5-dff6924d2b2f.local 61456 typ host\r\na=candidate:4 1 TCP 2105458943 65d476bf-60a6-4ec2-b9bb-99ca00f2c4ac.local 9 typ host tcptype active\r\na=candidate:5 1 TCP 2105524479 67213273-8df2-472b-82c5-dff6924d2b2f.local 9 typ host tcptype active\r\na=candidate:1 1 UDP 1685987327 67.171.207.78 61455 typ srflx raddr 0.0.0.0 rport 0\r\na=sendrecv\r\na=end-of-candidates\r\na=ice-pwd:ae2e1b7f176aaec61adcbce660136628\r\na=ice-ufrag:123f9481\r\na=mid:0\r\na=setup:active\r\na=sctp-port:5000\r\na=max-message-size:1073741823\r\n"}
2021/05/11 15:23:12 Received Answer.
2021/05/11 15:23:14 WebRTC: DataChannel.OnOpen
2021/05/11 15:23:14 ---- Handler: snowflake assigned ----
2021/05/11 15:23:15 Traffic Bytes (in|out): 0 | 2796 -- (0 OnMessages, 10 Sends)
May 11 17:23:17.000 [notice] Bootstrapped 14% (handshake): Handshaking with a relay
[OnionManager] progress=14
May 11 17:23:17.000 [notice] Learned fingerprint 2B280B23E1107BB62ABFC40DDCC8824814F80A72 for bridge 192.0.2.3:1 (with transport 'snowflake').
May 11 17:23:17.000 [notice] Bootstrapped 15% (handshake_done): Handshake with a relay done
May 11 17:23:17.000 [notice] Bootstrapped 20% (onehop_create): Establishing an encrypted directory connection
[OnionManager] progress=15
[OnionManager] progress=20
May 11 17:23:17.000 [notice] Bootstrapped 25% (requesting_status): Asking for networkstatus consensus
[OnionManager] progress=25
May 11 17:23:18.000 [notice] new bridge descriptor 'flakey' (fresh): $2B280B23E1107BB62ABFC40DDCC8824814F80A72~flakey [1zOHpg+FxqQfi/6jDLtCpHHqBTH8gjYmCKXkus1D5Ko] at 192.0.2.3
2021/05/11 15:23:20 Traffic Bytes (in|out): 15952 | 5878 -- (24 OnMessages, 18 Sends)
2021/05/11 15:23:20 WebRTC: Collecting a new Snowflake. Currently at [1/3]
2021/05/11 15:23:20 snowflake-d4f8fb7718eed98e connecting...
2021/05/11 15:23:20 WebRTC: DataChannel created.
2021/05/11 15:23:20 WebRTC: Created offer
2021/05/11 15:23:20 WebRTC: Set local description
2021/05/11 15:23:20 WebRTC: PeerConnection created.
2021/05/11 15:23:20 Negotiating via BrokerChannel...
Target URL: snowflake-broker.torproject.net.global.prod.fastly.net Front URL: cdn.sstatic.net
[OnionManager] Triggering Tor connection retry.
May 11 17:23:23.000 [notice] Closing no-longer-configured Socks listener on 127.0.0.1:39050
May 11 17:23:23.000 [notice] DisableNetwork is set. Tor will not make or accept non-control network connections. Shutting down all existing connections.
May 11 17:23:23.000 [notice] Opening Socks listener on 127.0.0.1:39050
May 11 17:23:23.000 [notice] Opened Socks listener connection (ready) on 127.0.0.1:39050
2021/05/11 15:23:23 copy loop ended
2021/05/11 15:23:23 ---- Handler: closed stream 3 ---
2021/05/11 15:23:25 Traffic Bytes (in|out): 25 | 33 -- (1 OnMessages, 1 Sends)
2021/05/11 15:23:30 Traffic Bytes (in|out): 33 | 25 -- (1 OnMessages, 1 Sends)
2021/05/11 15:23:30 BrokerChannel Response:
504 Gateway Timeout
2021/05/11 15:23:30 WebRTC: closing DataChannel
2021/05/11 15:23:30 WebRTC: closing PeerConnection
2021/05/11 15:23:30 WebRTC: Closing
2021/05/11 15:23:30 WebRTC: Unexpected error, no answer. Retrying...
2021/05/11 15:23:30 WebRTC: Snowflakes have melted Retrying...
2021/05/11 15:23:30 ConnectLoop: stopped.
2021/05/11 15:23:30 WebRTC: closing DataChannel
2021/05/11 15:23:30 WebRTC: closing PeerConnection
2021/05/11 15:23:30 WebRTC: DataChannel.OnClose
2021/05/11 15:23:30 WebRTC: Closing
2021/05/11 15:23:30 WebRTC: melted all 1 snowflakes.
2021/05/11 15:23:30 ---- Handler: end collecting snowflakes ---
2021/05/11 15:23:30 ---- Handler: discarding finished session ---
2021/05/11 15:23:30 Handler ended
2021/05/11 15:23:30 copying WebRTC to SOCKS resulted in error: io: read/write on closed pipe
2021/05/11 15:23:35 Traffic Bytes (in|out): 0 | 33 -- (0 OnMessages, 1 Sends)
```https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40039probetest is spinning with 100% CPU2021-10-28T16:35:01ZDavid Fifielddcf@torproject.orgprobetest is spinning with 100% CPUI just now (2021-04-05 16:22:49) noticed that probetest (#40013) on the broker is using 100% CPU:
```
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1486 root 20 0 765744 239348 9832 S 98.3 5.9...I just now (2021-04-05 16:22:49) noticed that probetest (#40013) on the broker is using 100% CPU:
```
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
1486 root 20 0 765744 239348 9832 S 98.3 5.9 91550:09 probetest
```
Judging by the CPU time of 91500 minutes and 9 seconds, it has been like this for about (91550 * 60 + 9) / 3600. / 24 = 63 days.meskiomeskio@torproject.orgmeskiomeskio@torproject.orghttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40033Fix goroutine leak in snowflake server2021-06-18T18:10:19ZCecylia BocovichFix goroutine leak in snowflake serverGot a monit alert this morning that the snowflake bridge is down. I tried resetting my webextension and got a `Could not connect to the bridge.` error.Got a monit alert this morning that the snowflake bridge is down. I tried resetting my webextension and got a `Could not connect to the bridge.` error.Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40015Investigate high retransmission rate of Snowflake2022-03-01T19:17:35ZCecylia BocovichInvestigate high retransmission rate of SnowflakeThe recent report on Snowflake distinguishability caught the fact that Snowflake has a very high rate of retransmissions during the handshake: https://arxiv.org/pdf/2008.03254.pdf
This was reported to cause latency and it sounds like a ...The recent report on Snowflake distinguishability caught the fact that Snowflake has a very high rate of retransmissions during the handshake: https://arxiv.org/pdf/2008.03254.pdf
This was reported to cause latency and it sounds like a bug. It could also apply to the rest of the connection after the handshake. We should look into this.Sponsor 96: Rapid Expansion of Access to the Uncensored Internet through Tor in China, Hong Kong, & Tibethttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/40005Use of http.DefaultTransport can cause the proxy and clients to hang indefini...2020-07-31T14:39:57ZCecylia BocovichUse of http.DefaultTransport can cause the proxy and clients to hang indefinitelyThe [http.DefaultTransport](https://golang.org/pkg/net/http/#RoundTripper) leaves `ResponseHeaderTimeout` unset by default. This means that the proxy and client that use this transport will hang indefinitely if they don't receive a respo...The [http.DefaultTransport](https://golang.org/pkg/net/http/#RoundTripper) leaves `ResponseHeaderTimeout` unset by default. This means that the proxy and client that use this transport will hang indefinitely if they don't receive a response from the broker. This happens rarely in practice, but we've seen it before with #29861. A better thing to do is set a long time out and log an error message if contacting the broker failed.
The timeout for clients should be at least as long as the broker's [ClientTimeout](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/master/broker/broker.go#L31) and likewise the proxies should time out after the broker's [ProxyTimeout](https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/blob/master/broker/broker.go#L32) would be triggered.Sponsor 28: Reliable Anonymous Communication Evading Censors and Repressors (RACECAR)Cecylia BocovichCecylia Bocovichhttps://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/issues/34435Update bug-reporting links for gitlab2021-07-09T18:26:25ZDavid Fifielddcf@torproject.orgUpdate bug-reporting links for gitlabAfter the [gitlab migration](https://lists.torproject.org/pipermail/tor-project/2020-June/002866.html) we need to update bug-reporting instructions.
* https://snowflake.torproject.org/#bugs<br>
https://gitweb.torproject.org/pluggabl...After the [gitlab migration](https://lists.torproject.org/pipermail/tor-project/2020-June/002866.html) we need to update bug-reporting instructions.
* https://snowflake.torproject.org/#bugs<br>
https://gitweb.torproject.org/pluggable-transports/snowflake-webext.git/tree/static/index.html?h=webext-0.3.1&id=d2a9a8fd136ac6bbba393de3d51fb7ca85e17b8a#n75
* https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/snowflake/-/wikis/home#reporting-bugs
I checked in snowflake.git and did not find anything that needs to be changed.Gabagaba@torproject.orgGabagaba@torproject.org