GitLab

I was staring at the snowflake logs after #40033 happened, and noticed a lot more io: read/write on closed pipe style errors than I was expecting. Looking at our proxy loop:

// Copy from one stream to another.
func proxy(local *net.TCPConn, conn net.Conn) {
	var wg sync.WaitGroup
	wg.Add(2)

	go func() {
		if _, err := io.Copy(conn, local); err != nil {
			log.Printf("error copying ORPort to WebSocket %v", err)
		}
		if err := local.CloseRead(); err != nil {
			log.Printf("error closing read after copying ORPort to WebSocket %v", err)
		}
		conn.Close()
		wg.Done()
	}()
	go func() {
		if _, err := io.Copy(local, conn); err != nil {
			log.Printf("error copying WebSocket to ORPort %v", err)
		}
		if err := local.CloseWrite(); err != nil {
			log.Printf("error closing write after copying WebSocket to ORPort %v", err)
		}
		conn.Close()
		wg.Done()
	}()

	wg.Wait()
}

If the client closes the connection, the bottom io.Copy will terminate and cause the other one to generate the error. If the OR connection closes: vice versa. However, when the client closes the connection, the bottom loop doesn't terminate because the bottom loop is reading from a KCP stream, not the WebSocket connection. So neither of these loops terminate until the OR connection times out (~20 minutes in a local test).

I'm not actually sure if this is bug or if it could cause performance problems. It means we're keeping goroutines and open connections around for longer than they need to be, but as far as I can tell we're not running out of memory on the bridge. At the very least it means that local.CloseRead and local.CloseWrite aren't doing anything here and will always generate errors.