MetricsPort: tor_relay_connections_total type confusion
Summary
For the MetricsPort
heros @mikeperry @dgoulet of these days.
I hope the current state does not end up in a stable version 0.4.7.11 which is planned for today I read.
tor_relay_connections_total
contains a mixture of counter and gauge values.
Different types should not be mixed in a single metric, this will cause confusion and incorrect usage/interpretation because the type declaration is not correct for all labels in that metric.
To quote from [1]:
As a rule of thumb, either the sum() or the avg() over all dimensions of a given metric should be meaningful (though not necessarily useful). If it is not meaningful, split the data up into multiple metrics. For example, having the capacity of various queues in one metric is good, while mixing the capacity of a queue with the current number of elements in the queue is not.
related:
- #40699 (closed)
- https://lists.torproject.org/pipermail/tor-relays/2022-October/020854.html
- https://prometheus.io/docs/concepts/metric_types/
- [1] https://prometheus.io/docs/practices/naming/
more reasoning: https://lists.torproject.org/pipermail/tor-relays/2022-October/020857.html
What is the current bug behavior?
Currently tor_relay_connections_total
contains a mixture of counters (state="created"
) and gauges (state="opened"
):
# HELP tor_relay_connections_total Total number of connections
# TYPE tor_relay_connections_total gauge
tor_relay_connections_total{type="OR",direction="initiated",state="created",family="ipv4"}
tor_relay_connections_total{type="OR",direction="initiated",state="created",family="ipv6"}
tor_relay_connections_total{type="OR",direction="received",state="created",family="ipv4"}
tor_relay_connections_total{type="OR",direction="received",state="created",family="ipv6"}
tor_relay_connections_total{type="OR",direction="initiated",state="opened",family="ipv4"}
tor_relay_connections_total{type="OR",direction="initiated",state="opened",family="ipv6"}
tor_relay_connections_total{type="OR",direction="received",state="opened",family="ipv4"}
tor_relay_connections_total{type="OR",direction="received",state="opened",family="ipv6"}
...
What is the expected behavior?
As per prometheus best practices gauges and counters should not be mixed in a single metric:
state="opened"
values are gauges (name should not end in _total
):
# TYPE tor_relay_connections gauge
tor_relay_connections{type="OR",direction="initiated",state="opened",family="ipv4"}
tor_relay_connections{type="OR",direction="initiated",state="opened",family="ipv6"}
tor_relay_connections{type="OR",direction="received",state="opened",family="ipv4"}
tor_relay_connections{type="OR",direction="received",state="opened",family="ipv6"}
state="created"
are accumulating counters (name ends with _total
), they never decrease:
# TYPE tor_relay_connections_total counter
tor_relay_connections_total{type="OR",direction="initiated",state="created",family="ipv4"}
tor_relay_connections_total{type="OR",direction="initiated",state="created",family="ipv6"}
tor_relay_connections_total{type="OR",direction="received",state="created",family="ipv4"}
tor_relay_connections_total{type="OR",direction="received",state="created",family="ipv6"}