Reduce or remove use of mutexes for Snowflake broker metrics
After some profiling the Snowflake broker, we found that goroutines do sometimes block on obtaining the mutex for metrics updates. In particular, (*IPC).ProxyPolls
and (*IPC).ClientOffers
are called very frequently and each of these request the i.ctx.metrics.lock
mutex multiple times.
- We should be able to update our metrics more efficiently by using sync/atomic.
- The prometheus metrics library we are using is thread safe, so many calls to functions/metrics from there don't need to surrounded by mutex access:
All exported functions and methods are safe to be used concurrently unless specified otherwise.
- For our own prometheus metrics, we use atomic integers, which means they are also thread safe
Edited by Cecylia Bocovich