Add nickname in userstats processing modules

We have an issue in counting metrics for Tor bridges where we are not accounting for nodes that use the same fingerprint but different nicknames (#40047 (closed)). From what it has been estimated we are undercounting bridge users by about 14,000 or 14%, since only 12.5% of Snowflake users are being counted. (onionoo#40022 (comment 2793712))

This is an issue with our assumption that the fingerprint is the ultimate identifier for a tor node. This MR introduce the nickname when counting clients.

While this won't be able to show different graph for each group of nodes with the same fingerprint but different nickname it would still correct our aggregated statistics that we publish on metrics.torproject.org

requested review from @gk

assigned to @hiro

changed the description

marked this merge request as draft

added 1 commit

• 0c903144 - Fix style checks

Compare with previous version

added 1 commit

• fde54619 - Update sql and sql tests

Compare with previous version

added 1 commit

• 73006c2b - Fix sql tests.

Compare with previous version

On a side note I just noticed the metrics-lib commit got updated ... not sure how or why.

To test the db changes (this has to be documented).

Install pgtap, postgresql-12, postgresql-server-dev-all:

$ sudo apt install pgtap postgresql-server-dev-all postgresql-12

Create necessary tables:

psql -d ipv6servers -f src/main/sql/servers/init-ipv6servers.sql 

psql -d userstats -f src/main/sql/clients/init-userstats.sql 

Run the ant target:

$ ant test-all-dbs

Please note that the servers tests are still having some issues, but since we are not touching the servers stats modules we can fix that in another PR (note to self: open a ticket and also add this in the builds - maybe).

added Needs Review label

added Needs Information label

@hiro: this is still marked as Draft: am I right that it is nevertheless the thing actually for review?

marked this merge request as ready

removed Needs Information label

mentioned in issue #40047 (closed)

@hiro is still trying to test that. Let's see how that goes first.

added Needs Information label and removed Needs Review label

added 1 commit

• aac3e3f7 - Add nickname column to imported table

Compare with previous version

added 1 commit

• 04538b45 - Add nickname also in merged table

Compare with previous version