Speedup ingestion of data into the DB
I have been thinking of ways to speedup the ingestion process at the DB. I have been thinking of a way to make the status table smaller while also being a bit smarter in how and where we aggregate data.
This issue aims to address the following:
-
Make the router status table smaller to avoid passing a large data object in memory and saving a large table (large in number of columns) -
Aggregate latest and first descriptors in materialized views to avoid sorting each time -
Make sure the family digest use sha1 for easy of use across metrics ecosystem
Edited by Hiro