Have a remote probe service to test snowflake proxy NAT compatability
Our attempt at solving #33666 (closed) has so far failed to produce any improvement to snowflake reachability for clients with symmetric NATs. My guess at the reason behind this is that the overwhelming majority of our proxies are web-based, and we don't have APIs available to us through Chrome or Firefox browsers to perform the NAT behaviour checks that our solution relies on. Other fixes such as poll adjustment rates and NAT type updates on failures to connect to restricted clients don't seem to be working, perhaps because the ratio of clients with symmetric NATs to other clients is too low or because the NAT type resets to unknown every 24 hours.
One way forward is to try setting up more dedicated standalone proxies with unrestricted NAT types and leaving proxies with unknown NAT types for clients we know don't have restricted NATs. However, this solution is limited and I'm worried about blocking down the line (for the same reason why setting up a TURN server isn't sustainable in the presence of censorship).
We don't have very many proxies with known unrestricted NATs:
Since we have the ability to manipulate docker container networking to reproduce a symmetric NAT topology, another option is to have proxies perform a probe test (similar to the one partially implemented in #32938 (closed)). However, this test would only have the proxies attempt to open a data channel. If they succeed, they can poll for the restricted client NAT bucket. If they fail, they'll poll for the unrestricted client NAT bucket.