new metrics graph showing number of 100mbit exits

added SponsorJ component::metrics/website priority::medium resolution::implemented status::closed type::project labels

Trac:
Cc: N/A to karsten

Replying to arma:

For the exit sponsoring plan, it would be good to make a periodic (e.g. daily) list of 100mbit+ exits. Specifically, these would be relays where:

Their bandwidth rate is at least 12500KB

Their advertised bandwidth is above some cutoff, say 5000KB.

Their exit policy allows at least 80, 443, 554, and 1755.

If we sorted this list by consensus weight, showing percentage of exit weights they are and providing links to atlas or some other individual page for each, we'd be doing even better.

By "periodic", do you mean updating that list periodically, or keeping lists of all past periods and letting users navigate between them? I have an idea for building the former, but not for the latter. Assuming you mean a single list of current 100mbit+ exits in the following.

How about we extend Atlas itself for this? It's the better fit than the metrics website, because it was specifically designed for displaying individual relay information vs. aggregated statistics.

The way how the extended Atlas would work is that you select a "stored query" of some sort instead of searching by nickname/fingerprint/address. Atlas would make sure that the three criteria you mentioned are included in the filter, and it would sort by consensus weight and show percentage of exit weights. And of course, if you click on a relay, you'd get the already known details page for that relay. In the near future, that details page might contain a graph with exit weight percentage over time.

The only downside of that plan is that it may take weeks until everything's implemented, and you probably want to have it really soon. That problem shouldn't prevent us from extending Atlas, because it's the better design IMO. But I can easily hack up a Python script that outputs the requested information. We could even run it in a daily cronjob and send its results to the tor-relays mailing list. We can shut it down once Atlas is ready.

Then we could imagine doing a graph over time of the number of relays in this list. Ideally we will be able to see it go up. :)

This is something for the metrics website. I'll make a sample graph later today or tomorrow and attach it here.

(I'm not quite sure what the advertised bandwidth cutoff should be. Ideally it would be quite high, except there might come a time where we have such excess capacity that some relays don't see enormous spikes. If they have the capacity, it's a feature that they don't see the spikes. So maybe we should pick a low cutoff to be flexible for the future.)

The suggested solution above would allow you to tweak the advertised bandwidth parameter in Atlas. Changing that parameter for the metrics website graph will be somewhat harder, but not impossible.

Replying to karsten:

By "periodic", do you mean updating that list periodically, or keeping lists of all past periods and letting users navigate between them? I have an idea for building the former, but not for the latter. Assuming you mean a single list of current 100mbit+ exits in the following.

Right, just the latest list is fine (for now ;)

But I can easily hack up a Python script that outputs the requested information.

Yes please.

Replying to arma:

Replying to karsten:

By "periodic", do you mean updating that list periodically, or keeping lists of all past periods and letting users navigate between them? I have an idea for building the former, but not for the latter. Assuming you mean a single list of current 100mbit+ exits in the following.

Right, just the latest list is fine (for now ;)

Okay. :)

But I can easily hack up a Python script that outputs the requested information.

Yes please.

Here we go. I extended the #6329 (moved) script to output what we're interested in here:

$ git clone git://git.torproject.org/metrics-tasks.git
$ cd metrics-tasks/task-6329/
$ ./tor-relays-stats.py -d
$ ./tor-relays-stats.py -x -t-1 -l
       CW    adv_bw   P_guard  P_middle    P_exit Nickname            Link                                                                           Exit Guard CC AS_num    AS_name
  1.1363%   0.5483%   0.0000%   0.0000%   3.4094% Unnamed             https://atlas.torproject.org/#details/2624AE0466BD02AFAF3F263D4361D79ABE0E7E05 Exit       se AS47155   ViaEuropa Sweden
  0.9842%   0.7437%   0.6839%   0.6838%   1.5849% lumumba             https://atlas.torproject.org/#details/24B1F63F7DF9F85D711864811CC401BE5EB5FB9A Exit Guard nl AS43350   NFOrce Entertainment BV
  0.9126%   0.3223%   0.0000%   0.0000%   2.7382% ahmiaTORproxy01     https://atlas.torproject.org/#details/35A9322E265EA3F07E76520D28E0C3BDD68C8F82 Exit       fr AS16276   OVH Systems
  0.9037%   0.7466%   0.6279%   0.6279%   1.4552% rainbowwarrior      https://atlas.torproject.org/#details/DB8C6D8E0D51A42BDDA81A9B8A735B41B2CF95D1 Exit Guard nl AS43350   NFOrce Entertainment BV
  0.8590%   0.5333%   0.0000%   0.0000%   2.5772% TerrorSquad         https://atlas.torproject.org/#details/8BAEB37A2E5F4A3A9FBB8F90D8901D714C52678B Exit       us AS23367   Genesis Adaptive, INC.
  0.8339%   0.7449%   0.5794%   0.5794%   1.3428% politkovskaja2      https://atlas.torproject.org/#details/B93DCC053D7F0472BB17A4514E06FE76D9FB714B Exit Guard nl AS43350   NFOrce Entertainment BV
  0.6254%   0.6931%   0.4346%   0.4346%   1.0071% politkovskaja       https://atlas.torproject.org/#details/7DCB5313B9541DD29C94BFDE0ADF91DC91D2CFE9 Exit Guard nl AS43350   NFOrce Entertainment BV
  0.5914%   0.3642%   0.4109%   0.4109%   0.9524% gurgle              https://atlas.torproject.org/#details/948CDA1CE63D2165567B81706CD8C0E9F8934A47 Exit Guard ca AS12093   University of Waterloo
  0.5843%   0.7660%   0.4060%   0.4060%   0.9408% chomsky             https://atlas.torproject.org/#details/253DFF1838A2B7782BE7735F74E50090D46CA1BC Exit Guard nl AS43350   NFOrce Entertainment BV
  0.5753%   0.5828%   0.0000%   0.0000%   1.7262% Unnamed             https://atlas.torproject.org/#details/505BD69565964F7B20D51A3FC4A4825BD93CA444 Exit       se AS47155   ViaEuropa Sweden
  0.5726%   1.2863%   0.3979%   0.3979%   0.9221% wau                 https://atlas.torproject.org/#details/0ECBAB33DD27A6DA5C1141B39F839F931F92334C Exit Guard ro AS39743   Voxility SRL
  0.5664%   0.4705%   0.0000%   0.0000%   1.6993% Unnamed             https://atlas.torproject.org/#details/AE5A97FA3591F133D8D039232CF0005088190C91 Exit       se AS47155   ViaEuropa Sweden
  0.5664%   0.3089%   0.3935%   0.3935%   0.9120% BostonUCompSci      https://atlas.torproject.org/#details/9D4D995AA745A3CAA6276AFAD505D3E4097AA075 Exit Guard us AS111     Boston University
  0.5530%   1.4749%   0.3842%   0.3842%   0.8904% sofia               https://atlas.torproject.org/#details/43691853EA556C21A77E006886A5DC579855F527 Exit Guard ro AS39743   Voxility SRL
  0.4894%   1.1065%   0.3401%   0.3401%   0.7881% gorz                https://atlas.torproject.org/#details/F3D4C7479F8789758A77FF61D2D8929311568394 Exit Guard ro AS39743   Voxility SRL
  0.3695%   0.4806%   0.2568%   0.2568%   0.5950% qwertyoruiop1       https://atlas.torproject.org/#details/3F6529905DC70EED873BFFC7172A889E131AAA85 Exit Guard nl AS29073   Ecatel Network
  0.3150%   0.4634%   0.0000%   0.0000%   0.9450% qwertyoruiop2       https://atlas.torproject.org/#details/4A6F047595008A194FB0AE916A24554D556658D7 Exit       nl AS29073   Ecatel Network
  0.2595%   0.2005%   0.1803%   0.1803%   0.4178% wagtail             https://atlas.torproject.org/#details/131B60B9AFE6AEA60042132D648798534ABEA07E Exit Guard ch AS13030   Init7 Global Backbone
  0.2461%   0.1938%   0.1710%   0.1710%   0.3962% jhtor               https://atlas.torproject.org/#details/FA02311AF49EB663CA2685A8604C403A9E10E6E3 Exit Guard gb AS16276   OVH Systems
  0.2353%   0.3774%   0.0000%   0.0000%   0.7060% anonnode20          https://atlas.torproject.org/#details/6BD3E034D42AB112A8ECE5B95FB904CFC7BFDCAD Exit       ua AS44820   Denis Pavlovich Semenyuk
  0.1485%   0.1964%   0.1032%   0.1032%   0.2392% SilentT             https://atlas.torproject.org/#details/12C1478D06E76B4A9D49301EC79276D3A7DE8332 Exit Guard us AS16276   OVH Systems
 12.3279%  12.6042%   5.3695%   5.3694%  26.2453% (total in selection)

On second thought, do we really want to send this output to a mailing list? You're so much more flexible using the script yourself, e.g., by aggregating by AS (add the -A flag) or by country (add the -C flag). Also, that enables you to play around with advertised bandwidth thresholds more easily (look for mentions of fast_exits_only in the script).

Trac:
Keywords: N/A deleted, SponsorJ added
Type: enhancement to project

When asked about the previous list and its lack of any of the usual top exit nodes (CCC ones, for example) Karsten told me that it was because of the weird requirements for ports 554 and 1755 (on top of the more usual 80 and 443).

We have torservers' ones (lumumba, wau, chomsky, sofia, gorz, rainbowwarrior, politkovskaja, politkovskaja2), that's nearly 40% of the above list.

Trac:
Username: delber

https://github.com/gsathya/metrics-tasks/commits/master

Add '--almost-fast-exit-relay' - Display only exits which have a bw rate between 80 Mbit/s and 95 Mbit/s, advertised bw between 2000kb and 5000kb and allow exiting to 80, 443 ports.

Add 'no more than 2 per /24' clause - Only 2 relays per /24 blocks are used. Figure out the network addr for a relay and store it in a dict(network_data) with a 'network'->'relay' mapping. Check if 2 relays are present in network_data for a particular network, if yes, then find the relay with the smallest exit probability and remove it from the list of all relays and add the current relay to the list of all relays.

I made a mistake in the commit msg, the mapping is actually 'network' -> ['relay', 'relay']. Running it with --almost-fast-exit-relay takes almost 9 secs. I wonder if I can optimize this even more. Thoughts?

Trac:
Status: new to needs_review

Trac:

I just attached six graphs as discussed with arma on #tor-dev this morning. Some meta stats: generating these graphs took 2 hours CPU time and 7:15 hours developer time.

gsathya, thanks for the patch above! I'll review and merge it tomorrow morning. (If you feel you want to tweak the commit message before it gets merged, please do.)

Replying to karsten:

gsathya, thanks for the patch above! I'll review and merge it tomorrow morning.
Thanks! (If you feel you want to tweak the commit message before it gets merged, please do.) Done.

gsathya, I looked at your patch. Here are some comments:

Your last commit, 787e3d4d, should be a "fixup" of 8e08f01. The two commits should get squashed before merging. (Feel free to rebase your branch before my next review if you want.)
When you call stats.get_relays() you don't pass options.almost_fast_exits_only, which means that you get all relays, not just the almost fast exits.
The --amost-fast-exit-relay option prints out relays almost failing both bandwidth requirements. It should instead print out relays failing at least one requirement. Similarly, the port check should exclude relays allowing all four ports, because those already meet the port requirement instead of almost meeting them.
The -f flag should become the short form of the --family option which is contained in another pending patch. Can you use -w instead (almost x in the alphabet..)?
You're looping over all or_addresses to implement the "no more than 2 relays per /24" requirement. This is problematic once relays can have two or more IPv4 addresses, because then you'll add relays twice or more often to the selection. Fortunately, relays won't have more than one IPv4 address very soon. Can you add a loud warning if a relay has more than one IPv4 address in that list?
Related to or_addresses and your ip.split(':') command, you're treating IPv6 addresses wrongly. IPv6 addresses for relays will be added very soon, so we should handle them correctly, i.e., ignore them entirely for the /24 requirement.
Just guessing, but I wonder if the script is slow, because you're using socket.inet_aton to extract the /24 of an address. Why do the hard math there instead of simply cutting off at the last dot?
Another idea for the performance problems might be that my ports-checking code is quite inefficient. When I implemented something similar in Java yesterday for the graphs, my VM was very unhappy with me. In particular, reject 1-65535 is the worst case for creating a temporary list containing 65535 ints. Want to look into rewriting that part more efficiently? Otherwise, I'd do it.
While you're working on this patch, can you change the bandwidth-rate requirement for --fast-exits-only from 12500 * 1024 to 95 * 125 * 1024, too? Maybe make that a separate commit.

Thanks a lot!

Trac:
Status: needs_review to needs_revision

Replying to karsten:

I just attached six graphs as discussed with arma on #tor-dev this morning. Some meta stats: generating these graphs took 2 hours CPU time and 7:15 hours developer time.

fast-exits-2months and almost-fast-exits-2months are great! Can we get them up on a metrics page somewhere, auto-updated several times a day?

Hmm, this was less painful than expected: https://metrics.torproject.org/fast-exits.html

Replying to karsten:

The --amost-fast-exit-relay option prints out relays almost failing both bandwidth requirements. It should instead print out relays failing at least one requirement. Similarly, the port check should exclude relays allowing all four ports, because those already meet the port requirement instead of almost meeting them.

Maybe this comment was badly phrased. Let me try again:

We defined three requirements---two bandwidth requirements and one port requirement---for relays to be considered "fast exits". We also relaxed all three requirements and defined relays meeting those requirements, but not the above requirements, as being "almost fast exits".

Your current code has two problems:

It only considers a relay as almost fast if it meets the relaxed bandwidth requirements but fails the original bandwidth requirements. In numbers, 80 < rate < 95 and 2000 < advertised < 5000. If a relay has a rate of 90 and advertises 6000, you wouldn't list it.
Once you rewrite the bandwidth requirements, you'll also want to rewrite the port requirement to check if all four ports are permitted. If a relay has rate >= 95 and advertised >= 5000, and supports all four ports (of which you only check two), you'd call it an almost fast exit, though it's in fact a fast exit.

Hope that helps.

Replying to karsten:

Replying to karsten: Your current code has two problems:

It only considers a relay as almost fast if it meets the relaxed bandwidth requirements but fails the original bandwidth requirements. In numbers, 80 < rate < 95 and 2000 < advertised < 5000. If a relay has a rate of 90 and advertises 6000, you wouldn't list it.

Oh. I thought an almost-fast-exit relay should satisfy both -

80 < rate < 95 and
2000< advertised < 5000. Sorry. So, it should be "80 < rate < 95 or 2000 < advertised < 5000"? Is this correct?

    if not ((80 * 125 * 1024 <= relay.get('bandwidth_rate', -1) <= 95 * 125 * 1024) or (2000 * 1024 <= relay.get('advertised_bandwidth', -1) <= 5000 * 1024)):
        continue

Once you rewrite the bandwidth requirements, you'll also want to rewrite the port requirement to check if all four ports are permitted. If a relay has rate >= 95 and advertised >= 5000, and supports all four ports (of which you only check two), you'd call it an almost fast exit, though it's in fact a fast exit.

Ok.

Trac:
Status: needs_revision to needs_information

Replying to gsathya:

So, it should be "80 < rate < 95 or 2000 < advertised < 5000"? Is this correct?

Not quite. That would accept rate = 90 and advertised = 1000, which it shouldn't. It's rather "(rate >= 80 and advertised >= 2000 and ports 80, 443) and not (rate >= 95 and advertised >= 5000 and ports 80, 443, 554, 1755)".

Trac:
Status: needs_information to needs_revision

Replying to karsten:

Replying to gsathya: Not quite. That would accept rate = 90 and advertised = 1000, which it shouldn't. It's rather "(rate >= 80 and advertised >= 2000 and ports 80, 443) and not (rate >= 95 and advertised >= 5000 and ports 80, 443, 554, 1755)".

Gotcha. Thanks! I'll merge this with delber's changes, and send you a patch.

Replying to karsten:

Hmm, this was less painful than expected: https://metrics.torproject.org/fast-exits.html

Thanks! This is really nice for keeping track of progress. The funders love it too.

Can you put up the most recent list of relays for each case too? That way we'll have an easy list we can give out, and also it will be easier for me to know which ones currently count so I can distinguish the 'almost counting' from the 'counting'.

Replying to arma:

Can you put up the most recent list of relays for each case too? That way we'll have an easy list we can give out, and also it will be easier for me to know which ones currently count so I can distinguish the 'almost counting' from the 'counting'.

Sathya is working on extending the #6329 (moved) Python script to print out these relays. You'd have to run that Python script locally though, instead of looking at a website.

Other than that, I think we should extend Atlas, not the metrics website, to list relays meeting or almost meeting the fast exit requirements. See my comment above.

If we really wanted these lists to show up on a website we could turn the #6329 (moved) script into a website. But once we have extended Atlas, we'd throw away that code. I'd rather want to avoid that.

What we could also do is set up a mailing list for the output of the #6329 (moved) script. We could then add a link to that mailing list archive to the metrics website.

Sathya made some good progress with the website version of Compass which will list fast exits and almost fast exits. But we were unsure how to exactly define "fast" and "almost fast". I hear Roger had some ideas there, too. We should really agree on a common understanding soon. How about this:

The set of fast exits has a. 95+ Mbit/s bandwidth rate, a. 5000+ KB/s advertised bandwidth, a. accepts ports 80/443/554/1755, and a. at most 2 relays per /24 network.
The set of almost fast exits has a. 80+ Mbit bandwidth rate, a. 2000+ KB/s advertised bandwidth, a. accepts ports 80/443, a. does not have (95+ Mbit/s bandwidth rate and 5000+ KB/s advertised bandwidth and allows ports 80/443/554/1755), and a. has as many relays per /24 network as there are.
The set of fast exits without network restriction has a. 95+ Mbit/s bandwidth rate, a. 5000+ KB/s advertised bandwidth, a. accepts ports 80/443/554/1755, and a. has as many relays per /24 network as there are.

The idea is that 1 is what the sponsor cares about, 2 contains the set of exits that we might turn into fast exits, and 3 gives an overview of network diversity.

If we agree on these requirements, I'll have to extend and re-run the tool producing the graph data, because right now the 80+ Mbit/s line has the "at most 2 per /24" requirement, too. That means that the line will slightly go up.

If this is the actual algorithm we'll use, 2.d. should be "is not in group 1". As you've specified it now, if there are 3 qualifying fast exits in a given /24, two of them make it into group 1 and zero of them make it into group 2. I think two of them should be in group 1 and the remaining one should be in group 2.

Other than that, sounds great!

Replying to arma:

If this is the actual algorithm we'll use, 2.d. should be "is not in group 1". As you've specified it now, if there are 3 qualifying fast exits in a given /24, two of them make it into group 1 and zero of them make it into group 2. I think two of them should be in group 1 and the remaining one should be in group 2.

Ah, the idea is that the third relay could be moved to a different /24 and then count as fast exit, too? Indeed, then it makes sense to only remove group 1 relays from the group 2 results.

Other than that, sounds great!

Great!

gsathya, I think the easiest fix is to combine FastExitFilter and SameNetworkFilter by adding same_network as new parameter to FastExitFilter. Otherwise, I don't see how we could implement the new case 2. The cases would then be something along this:

if options.fast_exits_only:
    filters.append(FastExitFilter(bandwidth_rate = 95 * 125 * 1024,
                                  advertised_bandwidth = 5000 * 1024,
                                  ports = [80, 443, 554, 1755],
                                  same_network = True,
                                  inverse = False))
if options.almost_fast_exits_only:
    filters.append(FastExitFilter(bandwidth_rate = 80 * 125 * 1024,
                                  advertised_bandwidth = 2000 * 1024,
                                  ports = [80, 443],
                                  same_network = False,
                                  inverse = False))
    filters.append(FastExitFilter(bandwidth_rate = 95 * 125 * 1024,
                                  advertised_bandwidth = 5000 * 1024,
                                  ports = [80, 443, 554, 1755],
                                  same_network = True,
                                  inverse = True))
if options.fast_exits_only_any_network:
    filters.append(FastExitFilter(bandwidth_rate = 95 * 125 * 1024,
                                  advertised_bandwidth = 5000 * 1024,
                                  ports = [80, 443, 554, 1755],
                                  same_network = False,
                                  inverse = False))

In the meantime, I'll go extend the graph code and re-run it to handle the new case 2.

Replying to karsten:

In the meantime, I'll go extend the graph code and re-run it to handle the new case 2.

This is done. (Updating Compass isn't.)

Compass now has lists of fast and almost fast exits. And we have graphs. AFAICS that concludes this ticket. Closing. Please re-open if I missed something.

Trac:
Resolution: N/A to implemented
Status: needs_revision to closed

Replying to karsten:

Compass now has lists of fast and almost fast exits. And we have graphs. AFAICS that concludes this ticket. Closing. Please re-open if I missed something.

In compass, the third option is listed as "Fast exits relays any network." It's not clear to me whether this means "Fast exits (plural), PLUS relays any network" or Fast exit relay. Having seen the code in git and the discussion above, I'm even less clear now on what that third option does. Maybe it's just the extra 's' appended to 'exits,' but could that third descriptor be clarified?

Trac:
Priority: normal to trivial
Status: closed to reopened
Cc: karsten to karsten, cypherpunks
Resolution: implemented to N/A
Component: Metrics Website to Compass

Moved to a new ticket: #9993 (moved)

Trac:
Component: Compass to Metrics Website
Status: reopened to closed
Resolution: N/A to implemented
Priority: trivial to normal

closed

mentioned in issue #7640 (moved)

mentioned in issue #7895 (moved)

new metrics graph showing number of 100mbit exits

Child items ...

Activity