Provide metrics data from the last 3--7 days via rsync
metrics-db provides compressed tarballs of its collected and sanitized files via rsync, so that others can copy them for their analyses or services. But these tarballs are only updated once per day, and updating them more often has performance implications on the metrics-db host. Also, having others update their services from potentially very large monthly tarballs has performance issues, too.
Can we make the collected and sanitized descriptors from the last, say, 3 to 7 days available via rsync? For a point of reference, in the first 8.5 days of December, metrics-db collected and sanitized 178,000 files being 2.4G in size. Are those numbers totally crazy for rsync?
We could implement this by having a script copy new descriptors to a directory that is then made available via rsync. The same script would also delete files that are older than 3--7 days.
This ticket is part of decoupling metrics-db from metrics-web and other metrics services. Right now, metrics-web uses the metrics-db output using a symbolic link the file system. That means metrics-db and metrics-web rely on running on the same host. We should change that to allow others to run their own metrics-web or or similar service.