deploy karma monitoring dashboard

Quote from TPA-RFC-33:

We will deploy a Karma dashboard to expose Prometheus alerts to operators. It features:

  • silencing alerts
  • showing alert inhibitions
  • aggregate alerts from multiple alert managers
  • alert groups
  • alert history
  • dead man's switch (an alert always firing that signals an error when it stops firing)

There is a Karma demo available although it's a bit slow and crowded, hopefully ours will look cleaner.

Note: we will need to come up with access permissions for the karma dashboard since some users will have the right to create silences through it. we could possibly keep a public readonly access, but members of TPA will need to be able to play with the silences.

  • deploy podman on prometheus.torproject.org
  • create user on server for running rootless container
  • generate a systemd unit using podman for running the image ghcr.io/prymitive/karma:v0.120 with the right env vars ALERTMANAGER_URI=https://prometheus.torproject.org/
  • figure out how we want to authenticate to karma and setup this authentication
  • expose karma as a sub-url of prometheus.torproject.org subdomain, karma.torproject.org, with the same authentication system as prometheus.tpo, proxied by apache
Edited by lelutin