Make various architecture improvements and modernizations

This is the parent ticket for further architecture improvements and modernization of the CollecTor code-base.

added component::metrics/collector metrics-2018 owner::metrics-team priority::medium severity::normal status::assigned type::enhancement labels

#8799 (moved) already describes the download modernization.

Further goals are

1. to replace the tarball-script with a Java module of CollecTor and
2. to make index.json as accurate as possible.

For 2) it might be useful to remove the separately configurable create-index module. The index should just be created after each (major) change, e.g. any download or sync module run.

The tarball creation could run as a continuous background process that is stops during sync and download runs, i.e. basically 'stays out of the way' when other modules run. It also relates to the index creation as it changes the 'archive' path. Maybe, trigger an index-run after a substantial change?

Other thoughts and general goals?

Addendum to comment:1:

• descriptor writing should only be performed by the 'o.t.c.persist' package
• One class of each module should extend CollecTorMain. It should be named Main, e.g. RelayMain or BridgeMain, and contain the code for structuring the modules work, i.e. no detailed functionality.
• The other classes of each module should provide the actual module functionality.

Addendum comment:1: Several modules perform some sort of cleaning, i.e. erase files based on their age. This should be offered by a utility class/package.

Added child ticket for CleanUtils.

This affects relay, bridge, exitlists, and torperf.

Tweak summary.

Trac:
Summary: CollecTor: architecture improvements and modernization to Make various architecture improvements and modernizations

Trac:
Keywords: N/A deleted, metrics-2018 added

Trac:
Owner: N/A to metrics-team
Status: new to assigned

mentioned in issue #21219 (moved)

mentioned in issue #22140 (moved)