triple stack (IPv4, IPv6, .onion) monitoring

We have just realized that we have a change of how monitoring is done between icinga and Prometheus.

Icinga is currently configured to check IPv4 addresses, explicitly. Each host is manually added in the configuration by their name and their IPv4 address. (There was actually an issue to ensure we have some monitoring for IPv6 and .onion sites as well, but it was never implemented, #29409 (closed) and #27634 (closed) )

Prometheus works differently: we put hostnames as targets, and monitor that. Therefore many checks happen over IPv6 instead of IPv4.

We should decide what we do with this. Normally, we would monitor both, but it's not clear to me how we should alert on those: what if a service works on IPv4 and not IPv6? is that critical? How about the opposite?

Then consider the fact that we, the tor project, maintain a third stack which is .onion services, and that should also be monitored.

Therefore, it seems what we need is not only dual-stack, but triple-stack monitoring.

For now, we'll keep the current approach of using whatever routing the monitor host picks. Perhaps the simplest way to do dual-stack monitoring is to have extra hosts (in high availability mode, #41643) that each have a single stack configuration (IPv4 or IPv6) and treat this as a "multiple vantage point alerting" problem (#41711).