document how to look up high cardinality labels authored by anarcat's avatar anarcat
Found while digging for issues with #42219
......@@ -1964,7 +1964,18 @@ long-term storage][] which suggests tweaking the
`storage.local.series-file-shrink-ratio`.
[This guide from Alexandre Vazquez][] also had some useful queries and
tips we didn't fully investigate.
tips we didn't fully investigate. For example, this reproduces the
"Highest Cardinality Metric Names" panel in the Prometheus dashboard:
topk(10, count by (__name__)({__name__=~".+"}))
The [api/v1/status/tsdb](https://prometheus.torproject.org/api/v1/status/tsdb) endpoint also provides equivalent
statistics. Here are the equivalent fields:
- Highest Cardinality Labels: `labelValueCountByLabelName`
- Highest Cardinality Metric Names: `seriesCountByMetricName`
- Label Names With Highest Cumulative Label Value Length: `memoryInBytesByLabelName`
- Most Common Label Pairs: `seriesCountByLabelValuePair`
[#41070]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/41070
[Grafana graph showing Prometheus disk usage]: https://grafana.torproject.org/d/000000012/prometheus-2-0-stats?orgId=1&refresh=1m&viewPanel=40&from=now-1y&to=now
......
......