new mail nodes (at least mx-dal-01) lacking postfix monitoring

eugeni and other mail nodes (e.g. alberti, lists-01, crm-01) have a mtail that parses postfix logs and generate metrics that we use to monitor queues and so on. we don't currently alert on those, AFAIK, but we do have a dashboard that is quite useful to diagnose such issues.

this would have come in handy when debugging recent bounce issues (#41993)... i had outlined that requirement in #40987 (comment 3088304) but it seems that requirement fell through the cracks.

so let's restore that monitoring to all new web nodes. this should probably be part of the base postfix profile, the same way it was (i assume?) part of the old postfix_tor profile when mail_processing was enabled...

maybe this was lost in the puppet refactoring as well (#40626 (closed))

marked this issue as related to #40987 (closed)

marked this issue as related to #40626 (closed)

changed milestone to %TPA-RFC-71: emergency email deployments, phase B

added Email Next Prometheus labels

assigned to @lavamind

assigned to @groente and unassigned @lavamind

assigned to @anarcat and unassigned @groente

mentioned in issue #41991 (closed)

i deployed the postfix mtail exporter class on all profile::postfix nodes. it can be turned off in hiera, but i'm not sure how to handle this...

@groente this used to be gated on the mail_processing flag, which would decide whether a node is its own MX or goes through eugeni to send mail...

right now, all nodes including profile::postfix seem to be in the "mailprocessing=true" category, so i've just set the setting to true by default but that should be revisited when more nodes are including the profile::postfix...

will let metrics collect for a while and check in a couple days.

added Needs Review label and removed Next label

i'm happy: we now have metrics for lists-01 too. @groente will cleanup the monitoring stuff soon, closing.

closed