service/prometheus.md

@@ -1342,37 +1342,35 @@ The receiver is configured below:
       - url: 'http://localhost:8098/'
 ```
 
-This URL, in turn, runs a simple Python script that just dumps to
-standard output all POST requests it receives, which provides us with,
-basically, a JSON log of all notifications sent through the
-Alertmanager. All logged entries since last boot can be seen with:
+This URL, in turn, runs a simple Python script that just dumps to a
+JSON log file all POST requests it receives, which provides us with a
+history of all notifications sent through the Alertmanager.
 
-    journalctl -u tpa_http_post_dump.service -b
+All logged entries since last boot can be seen with:
 
-You can see a prettier version of recent entries with the `jq`
-command, for example:
+    journalctl -u tpa_http_post_dump.service -b
 
-    journalctl -u tpa_http_post_dump.service -o cat -e  | grep '^{' | jq -C . | less -r
+This includes other status logs, so if you want to parse the actual
+alerts, it's easier to use the logfile in
+`/var/log/prometheus/tpa_http_post_dump.json`.
 
-Note that the `grep` is required because `journalctl` insists on
-bundling supervisor messages in its output, so we filter for JSON
-objects, basically. That said, this might just work as well:
+For example, you can see a prettier version of today's entries with
+the `jq` command, for example:
 
-    journalctl -u tpa_http_post_dump.service -o cat -e | jq -C . | less -r
+    jq -C . < /var/log/prometheus/tpa_http_post_dump.json | less -r
 
 Or to follow updates in real time:
 
-    journalctl -u tpa_http_post_dump.service -o cat -f | jq .
+    tail -f /var/log/prometheus/tpa_http_post_dump.json | jq .
 
-The top-level objects are actually notification groups, you can also
-restrict the output the actual, individual alerts with:
+The top-level objects are logging objects, you can also restrict the
+output the actual, individual alerts with:
 
-    journalctl -u tpa_http_post_dump.service -o cat -f | jq .alerts
+    journalctl -u tpa_http_post_dump.service -o cat -f | jq .args.alerts
 
-Note that journald rotates those logs pretty aggressively: last we
-checked, we only had 24h of history. Work should be done to write
-those to syslog or an actual file to keep them longer (and make
-parsing easier).
+Logs are automatically rotated every day by the script itself, and
+kept for 30 days. That configuration is hardcoded in the script's
+source code.
 
 See [tpo/tpa/team#42222](https://gitlab.torproject.org/tpo/tpa/team/-/issues/42222) for improvements on retention and more
 lookup examples.