Monitor anti-censorship www services with prometheus
In the anti-censorship team we currently monitor several services with sysmon. We recently discovered that sysmon doesn't seem to follow HTTP 301 redirects. This means that if a web service dies but the 301 redirect still works (e.g., BridgeDB is dead but its apache reverse proxy still works), sysmon won't notice.
Now that prometheus is running, we should fill this monitoring gap by testing the following web sites:
Our test should ensure that these sites serve the content we expect, e.g., make sure that bridges.tp.o contains the string "BridgeDB" in its HTML. Testing the HTTP status code does not suffice: if BridgeDB is down, the reverse proxy may still respond.
- monitor services in Nagios: BridgeDB, Snowflake, and GetTor
- deploy Prometheus's "blackbox exporter" for default bridges, which are external services
- delegate to (and train) the anti-censorship team the blackbox exporter configuration
- experiment with Prometheus's "alertmanager", which can send notifications if a monitoring target goes offline
- grant the anti-censorship team access to Prometheus's grafana dashboard.