Investigate why metrics-db sometimes takes longer than 5 minutes to run
metrics-db runs once every hour and shouldn't take longer than a few minutes. Ideally, we should run it twice per hour to catch consensuses published at :30 of an hour (see #5504 (moved)). That's only possible if it reliably finishes in under 30 minutes, ideally under 5 minutes.
My current idea why metrics-db takes so long is that it implements a quite crappy algorithm to provide the last 3 days of data via rsync: enumerate all files in the current rsync/ directory and store their names and last modified times in memory, iterate over our various output directories, copy files that are missing in rsync/, and delete the ones in rsync/ that are older than 3 days. The problem is the step where we iterate over output directories, because files are not deleted automatically after a given time. Any change needs to be implemented for relay descriptors, bridge descriptors, bridge pool assignments, exit lists, etc.
This task took 1.5 points so far and may take another 2.5 points for a real fix.