Investigate Unmeasured relays that get the Fast flag

I was looking at some consensuses today, and I noticed that there are many Unmeasured relays with the Fast flag, which doesn't seem right.

For example, in valid-after 2014-08-17 12:00:00 we have:

r k33ac R5oMCf+qgKvU/WE6nmB5/7/Vhlc E3tqtWZ92BE1vbOP4Wg1PKp241c 2014-08-17 02:11:59 50.38.49.173 443 9030
s Fast HSDir Running Stable V2Dir Valid
v Tor 0.2.4.23
w Bandwidth=20 Unmeasured=1

This is an investigative trac ticket, since I'm not sure if this behavior is a feature or a bug yet.

added 026-triaged-1 027-triaged-1-out component::core tor/tor priority::high resolution::duplicate status::closed type::defect labels

Looking at set_routerstatus_from_routerinfo() we have:

  rs->is_fast = node->is_fast =
    router_is_active(ri, node, now) &&
    !dirserv_thinks_router_is_unreliable(now, ri, 0, 1);

And in dirserv_thinks_router_is_unreliable() we have:

  if (need_capacity) {
    uint32_t bw_kb = dirserv_get_credible_bandwidth_kb(router);
    if (bw_kb < fast_bandwidth_kb)
      return 1;
  }
  return 0;

which is straightforward and relies on dirserv_get_credible_bandwidth_kb().

So, now let's dig into the interesting function dirserv_get_credible_bandwidth_kb(), which tells us:

/** Return the bandwidth we believe for assigning flags; prefer measured
 * over advertised, and if we have above a threshold quantity of measured
 * bandwidths, we don't want to ever give flags to unmeasured routers, so
 * return 0. */

The threshold is enforced here:

  /* Check if we have a measured bandwidth, and check the threshold if not */
  if (!(dirserv_query_measured_bw_cache_kb(ri->cache_info.identity_digest,
                                       &mbw_kb, NULL))) {
    threshold = get_options()->MinMeasuredBWsForAuthToIgnoreAdvertised;
    if (routers_with_measured_bw > threshold) {
      /* Return zero for unmeasured bandwidth if we are above threshold */
      bw_kb = 0;

Since, bwauths are enabled, I would have assumed that we are above that threshold and the bandwidth value for all unmeasured relays would be set to 0, however since unmeasured relays get the Fast flag it probably isn't the case.

And here is an attempt to emulate that threshold without being an authority:

$ grep Bandwidth consensus.z  | grep -v Unmeasured | wc -l
5616
$ grep Bandwidth consensus.z  | grep Unmeasured | wc -l
467

which means that there are 467 unmeasured relays and 5616 measured ones.

And we have:

  V(MinMeasuredBWsForAuthToIgnoreAdvertised, INT, "500"),

Trac:
Priority: normal to major
Milestone: Tor: 0.2.??? to Tor: 0.2.6.x-final

Trac:
Keywords: N/A deleted, 026-triaged-1 added

I assume the issue here is that only a minority of dir auths run bwauths, so the non-measuring ones are the ones that dictate the flags.

You're right, but I'm not even convinced if that is bad in the current network where the bwauths miss massive amounts of relays for extended amounts of time

This bug goes both ways: There are also measured relays deserving the fast flag but not getting it

Trac:
Milestone: Tor: 0.2.6.x-final to Tor: 0.2.7.x-final

Trac:
Status: new to assigned

Is there anything more to this bug? We have investigated and I think my explanation is right.

Marking triaged-out items from first round of 0.2.7 triage.

Trac:
Keywords: N/A deleted, 027-triaged-1-out added

Make all non-needs_review, non-needs_revision, 027-triaged-1-out items belong to 0.2.???

Trac:
Milestone: Tor: 0.2.7.x-final to Tor: 0.2.???

asn: ok to close?

So is the solution for this to run more bwauths?

Replying to teor:

So is the solution for this to run more bwauths?

Yes. And/or to implement #11327 (moved).

I'm going to close this one -- we can continue in #11327 (moved).

Trac:
Status: assigned to closed
Resolution: N/A to duplicate

Milestone renamed

Trac:
Milestone: Tor: 0.2.??? to Tor: 0.3.???

Milestone deleted

Trac:
Milestone: Tor: 0.3.??? to N/A

closed

mentioned in issue #15963 (moved)

moved to tpo/core/tor#12877 (closed)