Experiment with increasing conntrack table size on snowflake-01

During performance experimentation we (@linus and I) disabled connection tracking, suspecting it as a cause of high CPU use and because the conntrack table appeared to be dangerously close to being full. Let we re-enabled connection tracking to no apparent ill effect. Performance optimizations in snowflake-server were enough to bring the CPU use under control. The conntrack table still appears to be close to overflowing, but we are not actually seeing any "nf_conntrack: table full, dropping packet" kernel logs that would indicate an actual problem.

At this moment the conntrack table is 92% full (nf_conntrack_count / nf_conntrack_max = 240316 / 262144). It could be that this is normal and nothing to worry about. Let's try doubling the maximum number of entries and see if it reaches a new equilibrium. Currently we have:

# cat /proc/sys/net/netfilter/nf_conntrack_{count,max,buckets}
240316
262144
65536

I'm going to do this:

echo 131072 > /proc/sys/net/netfilter/nf_conntrack_buckets
echo 524288 > /proc/sys/net/netfilter/nf_conntrack_max

And meanwhile I'll run a tracking script to record nf_conntrack_count once per minute.