Visualize and track impact of customizable relay groups

Several years ago I knew the operators of something more than half the relays in the network by consensus weight.

Over the past few years though I think this statement is no longer true, in part because some relay operators have really ramped up their bandwidth (while the ones I know in person haven't), in part because the bwauths are centralizing the network towards certain ISPs in Europe, and in part because the set of relay operators is (or maybe rather, appears to be) expanding.

The churn in the network, and the unpredictability of consensus weights, make it so that even if I convince myself that I know 50% of the weights this week, that doesn't help me learn much about whether I'll know 50% of the weights next week. The lack of a framework for annotating relays discourages me from engaging with the question of "what percentage of today's Tor network do I know?"

One option for how to limit the risk of malicious relay operators is a Tor network where we have to vet every relay operator, or a Tor network where we have to vet all of the ones that get to be a guard or an exit (or an hsdir or etc). Another option is to limit the influence that any given relay or relay family can have on the network.

We've had some variations of this idea over the years but most of the things we've actually built limit individual relays, rather than families or groups. For example, before we added the bwauth design, we limited the weight of any single relay to 10000. I opened tpo/core/tor#40007 (moved) for how we should think through being able to limit the total network fraction that a given family makes up.

But ultimately the trouble is that we can't easily tell whether relays or relay families are operated by the same individual. And our workaround for scaling, where we encourage people to run multiple Tor instances to fill huge pipes, still makes sense, but also it makes the "can't easily tell" problem worse, because it incentivizes and normalizes the world where both good people and bad people are running dozens or even hundreds of relays.

Instead, I suggest an approach where the total influence from relays we don't "know" is limited to some fraction, like 50% or 25%.

With this approach, so long as we are approximately correct in labeling the relays that we know, we can be assured of some security parameters for the network as a whole.

One concrete tool that would help me track "what fraction of the network I know" is some sort of "relay grouping" or "relay teams" functionality, where I can click "I know these relays" or "I don't know those relays" and then it shows me not-yet-tagged relays, and helps me visualize my group of relays over time compared to the whole. I'm imagining something like the Nos Oignons tools:
https://nos-oignons.net/Services/index.fr.html
but with an easy way to manage and track my own customizable relay set.

With such a tool, I could assess how much of the network I know today, as well as follow how that fraction is changing/degrading over time. And other people could do the same thing.

In an ideal world, we can each tag each relay as we see fit, and then we'll be able to draw other useful conclusions, like "Roger's relay list is actually a subset of Mo's relay list" or "no one of us knows more than 20% of the network, but together we know 40% of the network" or "if we bumped out all the OVH relays, that 40% would go up to 70%".

Now, this mechanism for tagging relays is just a building block for how we might make this 'known relay set' idea impact path selection / relay weights. One idea I've heard for putting it all together, which sounded pretty good to me, would be to have a team of people who each track their knowledge of the network, and we'd use some aggregated view of that team when deciding which relays are in which category. "Known by anybody on the team" could be a great first approach, and later if needed we could add more nuance, like requiring redundancy for extra large relays, or heck, some sort of Advogato-style max-flow-min-cut reputation graph.