Deploy further snowflake-server performance improvements 2022-09-24
The snowflake-01 bridge is getting close to its memory limits again after yesterday's deployments (#40173 (closed), #40175 (closed)). The little dip marked in the graph below, I investigated it, and it occurred during a time when the memory was almost 100% full. You can also tell the server is reaching its limits because the send and recv traces become a little bit decorrelated. For comparison, currently, at the right edge of the graph, RAM use is 85%.
Profiling yesterday revealed two major contributors to memory usage:
io.Copy buffers and client send queues, each of which accounts for about 25%
of the total memory used by the process.
I have a branch that is meant to resolve the
io.Copy buffer one.
It exposes the
io.WriterTo interface of
which will prevent
io.Copy from allocating a buffer per client.
I also put in a minor optimization to
the benefit of which turns out to be tiny,
but there's no reason not to do it.