GitLab

Implements #30260.

Part of #33655.

Implements #33549.

This patch focuses on providing more context in log messages and unify
the use of log levels. It also adds counters and timestamps as far as
they are being used in log messages. And in very few cases it contains
minor changes like using try-with-resource when we're touching that
code anyway. But most other refactorings are left as future work.

Fixes the first part of #24542. The second part will be to improve
logging of the web server parts.

since no regex involved, replace() should be faster than replaceAll() since no Pattern.compile().matcher() will be needed behind the scenes

Turn absolute links into relative links.

Fixes #27925.

This reverts commit b0165cb0.

Turns out that worst case query time goes up from 3 to 58 seconds with
the new query, which is unacceptable. We'll have to analyze and
improve the query on the production database before merging this
change.

We're now counting the hours for which we have statuses and exit
lists. More precisely, we're truncating consensus valid-after times
and exit scan times to the hour and counting how many distinct hours
we saw during the requested time. If we're missing 18 or more hours of
statuses or exit lists we're printing out a note in the summary.

Fixes #31071.

The default listen host is still 127.0.0.1, but now it's possible to
configure a different listen host, like 0.0.0.0, which can facilitate
local testing.

Implements part of #31649.

Configuration is held in the .gitlab-ci.yml file.

Fixes: #31401

Includes a metrics-lib upgrade to 2.6.2 and an upgrade to the latest
metrics-base commit 256e619.

Implements part of #31326.

Reported by nusenu, changes suggested by irl.

Fixes #30415.

Implements #28859.

Implements #28584.

The changes in this commit are mainly driven by the following database
changes:

 - Fingerprints are no longer stored repeatedly as strings in the
   statusentry and exitlistentry tables but instead moved to their own
   fingerprint table and referenced by identifier. This is a step
   towards normalization that ideally reduces database size a lot by
   itself.
 - The statusentry table no longer contains the raw status entry but
   instead only those parts that are required for searching and
   displaying results. This includes nickname and exit information,
   where the former is not stored as string but as reference into its
   own nickname table.
 - There is a new table specifically for lookups which returns all
   relevant fingerprint identifiers for a date and an IP address
   prefix. The new search function based on this new table performs as
   little table joining as necessary and leaves some of the filtering
   and joining to the servlet.

More details can be found in the exonerator2.sql file, which also
serves as migration script from the existing schema to the new one.

Implements #27356.