service/prometheus.md

@@ -2050,6 +2050,30 @@ statistics. Here are the equivalent fields:
 [This guide from Alexandre Vazquez]: https://alexandre-vazquez.com/how-it-optimize-the-disk-usage-in-the-prometheus-database/
 [sawtooth wave]: https://en.wikipedia.org/wiki/Sawtooth_wave
 
+### Out of disk space
+
+The above procedure is useful to deal with "almost out of disk space"
+issues, but doesn't resolve the "actually out of disk space"
+scenarios.
+
+In that case, there is no silver bullet: disk space must be somehow
+expanded. When Prometheus runs out of disk, it starts writing a log of
+log files, so you might be able to get away with removing
+`/var/log/syslog` and `daemon.log` in an emergency, but fundamentally,
+more disk needs to be allocated to Prometheus.
+
+ 1. First, stop the Prometheus server:
+
+        systemctl stop prometheus
+
+ 2. Remove, compress logs, or add a new or grow a volume to make room
+
+ 3. Restart the server:
+
+        systemctl start prometheus
+
+You want to keep an eye on the [disk usage dashboards](https://grafana.torproject.org/d/zbCoGRjnz/disk-usage).
+
 ### Default route errors
 
 If you get an email like: