prometheus should monitor for obsolete packages (warning)
in #41639, we have all sorts of metrics deployed, but we somehow missed the important dsa-check-packages
script that's hooked into nagios.
there's some of those metrics available thanks to the apt_info.py
script in prometheus-node-exporter-collectors, but it doesn't have all of it. namely, it has:
# /usr/share/prometheus-node-exporter-collectors/apt_info.py
# HELP apt_upgrades_pending Apt packages pending updates by origin.
# TYPE apt_upgrades_pending gauge
apt_upgrades_pending{origin="",arch=""} 0
# HELP apt_upgrades_held Apt packages pending updates but held back.
# TYPE apt_upgrades_held gauge
apt_upgrades_held{origin="",arch=""} 0
# HELP apt_autoremove_pending Apt packages pending autoremoval.
# TYPE apt_autoremove_pending gauge
apt_autoremove_pending 21
# HELP node_reboot_required Node reboot is required for software updates.
# TYPE node_reboot_required gauge
node_reboot_required 0
dsa-check-packages has this:
# dsa-check-packages
OK: 618 ok, 4 rc
618 packages current.
4 packages removed but not purged: linux-image-6.1.0-18-amd64, linux-image-6.1.0-17-amd64, linux-image-6.1.0-21-amd64, linux-image-6.1.0-20-amd64
|obs_loc=0;1;5;0 outdated=0;1;5;0 current=618;;;0 obs_ign=0;;;0 rm_unprg=4;;;0 hold=0;;;0 prg_conf=0;1;;0
that is:
-
held packages ( apt_upgrades_held
) -
obsolete packages, including a way to exclude packages (not covered!) -
outdated packages ( apt_upgrades_pending
) -
current packages (not covered!) -
removed packages with leftover configuration ("removed but not purged", not covered) -
purged packages with leftove configuration (!?) (not covered) -
pending upgrades but that apt upgrade won't upgrade (see for example Ubuntu phased upgrades or pinned packages?) (prometheus-alerts#19) -
add an ignore list
the "obsolete packages" is the most important, as those are packages that might not be covered by security upgrade and a manual configuration...
all of those should be warnings, of course.
we should make this as a standalone script outside of apt_info.py
at first considering the time it took to fix that thing, but see also #41355 (closed).
Edited by anarcat