Expose broker metrics for Prometheus?
We are already exposing metrics but we don't have convenient tooling that turns these metrics into charts that are easy to explore and update automatically.
We could solve this problem by exposing these metrics in a format that our Prometheus instance can scrape. I recently did that for bridgestrap, over at tpo/anti-censorship/bridgestrap#4 (closed). It's not a lot of work but the downside is pulling in yet another semi-complex dependency.
If we were to implement this, here's how it would work:
- The broker serves a new page, e.g. /prometheus-metrics
- We can use Prometheus's Go client library to deal with metrics.
- Throughout the code, we can update metrics like this:
metrics.NumUnrestrictedNatProxies.Inc()
Does the above sound sensible? If so, I can implement a prototype.