The sync-process will available for the modules relaydescs, bridgedescs, exitlists, and torperf.
The additional functionality should be generalized as far as possible and module dependent functionality should be part of the module's code.
== Configuration ==
## Configuration
1. General settings:
Add the properties
`SyncRelayDescriptors`, `SyncBridgeDescriptors`, `SyncExitLists`, and `SyncTorperfFiles`
...
...
@@ -16,20 +16,20 @@ The additional functionality should be generalized as far as possible and module
The entire substructure of 'recent' will be fetched, i.e. `recent/exit-lists/*` for exitlists, `recent/relay-descriptors/**/*` for relaydescs, and `recent/bridge-descriptors/**/*` for bridgesdescs.
1. Backup of replaced local files:
There won't be a backup of replaced local files.
== Fetching and Merging ==
## Fetching and Merging
If `Sync*` has the value `NoSync`, nothing is done. `SyncOnly` will not start the module and immediately begin fetching from the instances configured in `SyncSources*`. `Sync` will first run the module and then begin to sync.
=== Processing ===
a. Retrieve descriptors from the CollecTor instances defined in `SyncSources*`. These descriptors are stored in `SyncFolder` under the host part of the instance's url, e.g. {{{my-sync-folder/collector.torproject.org/recent/exit-lists}}} for exitlists from the main instance.
### Processing
a. Retrieve descriptors from the CollecTor instances defined in `SyncSources*`. These descriptors are stored in `SyncFolder` under the host part of the instance's url, e.g. `my-sync-folder/collector.torproject.org/recent/exit-lists` for exitlists from the main instance.
b. Following retrieval the fetched descriptors are examined:
i. discard descriptor files that do not contain what they should (see comment:11) and log a warning with sync-source info and reason (see criteria).
i. copy valid descriptors (see criteria) without a pre-existing local copy to the local `*OutDirectory` (cf. [https://gitweb.torproject.org/collector.git/tree/src/main/resources/collector.properties collector.properties]) and 'recent' structure.
i. copy valid descriptors (see criteria) without a pre-existing local copy to the local `*OutDirectory` (cf. [collector.properties](https://gitweb.torproject.org/collector.git/tree/src/main/resources/collector.properties)) and 'recent' structure.
i. if there is a local copy already, decide which copy to keep (see criteria).
I. local copy is kept, log debug message with source and reason.
I. local and fetched are identical, log debug message with source and reason.
I. Maybe later: fetched copy should replace local descriptor. Copy fetched descriptor to local `*OutDirectory` and 'recent'. In all cases log debug message with source and reason.
== Replacement criteria ==
## Replacement criteria
As the replacement criteria are not fully defined yet and it is very likely that there will be more criteria in future a modular/pluggable approach seems useful, i.e.:
1. define `KeepCriterium` and `ReplaceCriterium` interfaces
1. register implementing classes with CollecTor in order to facilitate the selection steps described above.
