now that [certificate transparency](https://certificate.transparency.dev/) (CT) is a thing, we should probably start monitoring those. specifically, because major web browsers now *require* CAs to publish their certs in CT, there's a good chance that a hostile actor who manages to generate a cert for us would show up on that radar. (it wouldn't keep a rogue CA from generating a fake cert though, as they could, in theory, disobey those policies at some point.) As a side note, Chrome and Safari currently [enforce CT for HTTPS sites](https://en.wikipedia.org/wiki/Certificate_Transparency#Mandatory_certificate_transparency) (but not Firefox) as well. The goal here is to have an alert in monitoring (typically nagios/icinga right now) when a rogue certificate gets issued so we can act quickly on it. What to do next is out of scope, but should also be considered eventually. This is actually a surprisingly hard problem, because the CT logs are huge and constantly changing, so we can't just "run a log and watch it". The tooling around this isn't well settled either. Finally, we need to consider that we *do* issue certificates regularly and those shouldn't trigger an alert. # Known solutions * [cert spotter](https://sslmate.com/certspotter/) is both a commercial service from sslmate.com and a [free software log monitor](https://github.com/SSLMate/certspotter) written in Golang, outputs matching certs to stdout, doesn't seem actively maintained * [fetchallcerts.py](https://git.sunet.se/catlfish.git/tree/tools/fetchallcerts.py) is a homegrown script from @linus that is a proof of concept that writes a JSON representation of the merkle tree, zips up matching certs and warns about inconsistencies in the log * Let's Encrypt runs [ct-woodpecker](https://github.com/letsencrypt/ct-woodpecker) which involves running a full log and is more useful for actual CAs that want to monitor their own things * [DSA](https://dsa.debian.org/) uses [this nagios plugin](https://salsa.debian.org/dsa-team/mirror/dsa-nagios/-/blob/master/dsa-nagios-checks/checks/dsa-check-ct-logs) to monitor certspotter, and it *does* check for the existing cert bundle, see [this YAML config](https://salsa.debian.org/dsa-team/mirror/dsa-nagios/-/blob/5c73a99322881ec2ea6b4fe7ae7ae188aebfa19a/config/nagios-master.cfg#L2110) for how it's called (` remotecheck: "/usr/lib/nagios/plugins/dsa-check-ct-logs --domain debian.org --dir /srv/letsencrypt.debian.org/var/result/ --cert-bundle /etc/ssl/ca-global/ca-certificates.crt --subdomains --ignore-re '.*\\.acc\\.umu\\.se' --ignore-fp 90b1c027ff49c22e1dfbde6dcd4e3ef99d795ffe02e61e5ef3850896a33a430b"`) * [ct-sans](https://git.cs.kau.se/rasmoste/ct-sans) and (a less-deps rewrite) [ct](https://gitlab.torproject.org/rgdd/ct) * [certstream-python](https://github.com/CaliDog/certstream-python), client for [certstream.calldog.io](https://certstream.calidog.io/) * [scrape-ct-log](https://github.com/mpalmer/scrape-ct-log) claimed to be the "fastest" in [this post](https://www.hezmatt.org/~mpalmer/blog/2024/01/16/pwned-certificates-on-the-fediverse.html) * [ct-exporter](https://github.com/Hsn723/ct-exporter) - prometheus exporter, not sure what it does In a duplicate of this issue (#33602), I tested cert spotter: > it's a debian package since buster. i ran a test locally, and it's basically: > > ``` > sed 's/ /\n/g;/^#/d;/^ *$/d' letsencryt-domains/domains | sort | certspotter -watchlist - > ``` > > the key trick however, is to *not* warn *when* a new cert is renewed. therefore we would need to be somewhat clever and recognize our own certificates in there and filter those out.