Loosen restrictions on message sizes in WebSocket server
ahf couldn't bootstrap beyond 25% when running his own client, broker, and WebSocket server (i.e., not using the public infrastructure). I asked him to try relaxing the message size limit in server.go:
-const maxMessageSize = 64 * 1024
+const maxMessageSize = 10 * 1024 * 1024
This enabled him to bootstrap to 100% at least once, but "it still doesn't work most of the times i bootstrap from a clean tor instance."
What I suspect is happening is that the browser proxy is sending WebSocket messages larger than 64 KB, which is causing the WebSocket server to error and tear down the connection. How much larger than 64 KB, I don't know. The underlying websocket package returns an error like
"frame payload length of %d exceeds maximum of %d"
but we currently throw away that error message as a precaution until we've [ticket:21304 audited error logs] to ensure that IP addresses don't appear.
As an alternative to allowing larger messages at the server, we could try to ensure that proxies don't produce such over-large messages. Here, in onClientToRelayMessage
, we could break recv
into 64 KB chunks before pushing them onto @c2rSchedule
. In proxy-go, I suspect we are succeeding by accident: the code uses io.Copy, which must use an internal buffer smaller than 64 KB.
Taking a longer view, there's no good reason for a message size limit to exist at all. It stems from a time when I was naive in Go and didn't know how to provide an implementation that didn't buffer the entirety of each message in memory. The reason I wrote my own WebSocket implementation was that the other one that existed at the time, golang.org/x/net/websocket, had no limits on message size at all, and you could trivially DoS it (out of memory) by sending a 1 TB message. (Looks like this got fixed in 2016.) A good solution would be to rewrite our WebSocket library to provide a streaming interface without message buffering, so to investigate whether other WebSocket libraries like https://godoc.org/github.com/gorilla/websocket can do it.