Skip to content

Deploy further snowflake-server performance improvements 2022-09-24

The snowflake-01 bridge is getting close to its memory limits again after yesterday's deployments (#40173 (closed), #40175 (closed)). The little dip marked in the graph below, I investigated it, and it occurred during a time when the memory was almost 100% full. You can also tell the server is reaching its limits because the send and recv traces become a little bit decorrelated. For comparison, currently, at the right edge of the graph, RAM use is 85%.

bandwidth on network interface

Profiling yesterday revealed two major contributors to memory usage: io.Copy buffers and client send queues, each of which accounts for about 25% of the total memory used by the process. I have a branch that is meant to resolve the io.Copy buffer one. It exposes the io.WriterTo interface of smux.Stream through SnowflakeClientConn, which will prevent io.Copy from allocating a buffer per client. I also put in a minor optimization to ClientMap.SendQueue, the benefit of which turns out to be tiny, but there's no reason not to do it.

dcf/snowflake@a16133ff...4d4fad30

/cc @linus