Too many open files error at probetest
I took a look at the NAT probetest logs this morning after the alert yesterday. It doesn't look like probetest is stuck, but I did notice a familiar message in the logs:
2022/01/29 21:02:32 http: Accept error: accept tcp [scrubbed]: accept4: too many
open files; retrying in 5ms
And when I run a proxy, I frequently get http timeout errors, like due to this issue on the server.
We ran into this recently with the load balanced bridge (#40095 (comment 2772325)) and previously with the broker (#31425 (closed)).
The NAT probetest is run with sv, and it's run
file contains the following:
#!/bin/sh -e
export GOMAXPROCS=1
setcap 'cap_net_bind_service=+ep' /usr/local/bin/probetest
exec ip netns exec net0 /usr/local/bin/probetest --acme-hostnames snowflake-broker.bamsoftware.com,snowflake-broker.freehaven.net,snowflake-broker.torproject.net --acme-email dcf@torproject.org --acme-cert-cache /home/snowflake-broker/acme-cert-cache --addr :8443 2>&1