Make it easier for relay operators to find their observed bandwidth

Hi tom, thanks for #20372 (moved), makes a big difference to relay operators.

I was answering another relay operator question today, and realised I needed the observed bandwidth. Because they were measured high, but then limited by the observed bandwidth.

Is that the sort of thing we can put on consensus health? Under the fingerprint? (Because it's set by the relay, not an authority.) Or another row? Also, should we show the consensus weight?

Or is that just too confusing, and I should direct them to use Atlas, and hover over the bandwidth heading? (If it works, which it doesn't on high security tor browser.)

added component::metrics/consensus health owner::tom priority::medium severity::normal status::new type::enhancement labels

So depictor (consensus-health) uses stem to get its info, so it's really easy for me to include any information here: https://stem.torproject.org/api/descriptor/router_status_entry.html It's really difficult to get anything else - downloading descriptors for the network would take a lot of time, just downloading the votes and consesnsuses takes 10+ minutes.

I'm a little confused by the terms you're using. When you say "observed bandwidth" do you mean the relay's 'advertised bandwidth' a relay operator puts in the config, the unit-less values the bwauths calculate and vote on, or a third thing I'm not recalling? (And if it is the third thing, where does that get exposed: Consensus, MicroConsensus, Descriptor, Microdescriptor, or ExtraInfo?) And then the same question for "consensus weight".

Looking closer at the stem documentation, I might have done things wrong in #20372 (moved) actually. Adding Damian to confirm the below is correct:

• On a vote .measured is the bwauth's vote for the relay's (unitless) bandwidth value
• On a consensus .measured is the average* of the bwauths votes for the relay's (unitless) bandwidth value
• On a vote or consensus .bandwidth is the relay's advertised bandwidth (and is what teor wants available)

*not really average but let's pretend it is for simplicity

(Currently, the value on each Authority is the .measured but the value on the consensus is .bandwidth - which would be wrong as the consensus would be reporting the self-reported advertised bandwidth...)

Trac:
Cc: tom to tom, atagar

Replying to tom:

So depictor (consensus-health) uses stem to get its info, so it's really easy for me to include any information here: https://stem.torproject.org/api/descriptor/router_status_entry.html It's really difficult to get anything else - downloading descriptors for the network would take a lot of time, just downloading the votes and consesnsuses takes 10+ minutes.

I think I might be asking for the "bandwidth (int) -- bandwidth claimed by the relay (in kb/s)", but I am not sure how it is sourced.

I'm a little confused by the terms you're using. When you say "observed bandwidth" do you mean the relay's 'advertised bandwidth' a relay operator puts in the config, the unit-less values the bwauths calculate and vote on, or a third thing I'm not recalling? (And if it is the third thing, where does that get exposed: Consensus, MicroConsensus, Descriptor, Microdescriptor, or ExtraInfo?)

By "observed bandwdith" I mean the "bandwdith-observed" figure from the descriptor "bandwdith" line: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

I'd like it more easily available because it's one of the two figures that an operator doesn't set themselves - the other is the measured bandwidth.

And used in conjunction with the advertised bandwidth to calculate the bandwidth in the consensus "w" line: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2268

(That figure is useful, but more confusing, because it's sometimes the advertised bandwidth, and sometimes the observed bandwidth.)

And then the same question for "consensus weight".

Whatever Atlas uses for consensus weight in the relay details page. I assume it's measured bandwidth from the consensus w line, but I'm not sure.

If it is, it's already there in the consensus column of the relay flags table.

https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2274

Looking closer at the stem documentation, I might have done things wrong in #20372 (moved) actually. Adding Damian to confirm the below is correct:

• On a vote .measured is the bwauth's vote for the relay's (unitless) bandwidth value
• On a consensus .measured is the average* of the bwauths votes for the relay's (unitless) bandwidth value
• On a vote or consensus .bandwidth is the relay's advertised bandwidth (and is what teor wants available)

Not quite, I think it's the minimum of advertised and observed, which confuses a lot of operators, because sometimes it's controlled by the torrc, and other times by their relay's self-reported performance: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

*not really average but let's pretend it is for simplicity

(low-median)

(Currently, the value on each Authority is the .measured but the value on the consensus is .bandwidth - which would be wrong as the consensus would be reporting the self-reported advertised bandwidth...)

That's strange, because all the figures in the table appear to be a low-median (consensus .measured) to me.

Replying to teor:

Replying to tom:

So depictor (consensus-health) uses stem to get its info, so it's really easy for me to include any information here: https://stem.torproject.org/api/descriptor/router_status_entry.html It's really difficult to get anything else - downloading descriptors for the network would take a lot of time, just downloading the votes and consesnsuses takes 10+ minutes.

I think I might be asking for the "bandwidth (int) -- bandwidth claimed by the relay (in kb/s)", but I am not sure how it is sourced.

So I'm pretty sure stem's documentation is wrong.

On my relay (rittervg)'s consensus.routers['C0EDB08D7540D1DD3CA69809ED17D979F51B66E3'] https://atlas.torproject.org/#details/C0EDB08D7540D1DD3CA69809ED17D979F51B66E3 RouterStatusEntry:

• .is_unmeasured is false
• .measured is empty
• .bandwidth is the value from the consensus w line (the agreed upon value by the bwauths)

If I manually pull all the server descriptors (which I'm trying to avoid but I guess I could) I get the following new values:

• .observed_bandwidth - 1434333
• .average_bandwidth - This is the value Atlas displays under 'Advertised Bandwidth'
• .burst_bandwidth - 1536000

My relay's torrc says

• RelayBandwidthRate 1250 KBytes
• RelayBandwidthBurst 1500 KBytes

I'm a little confused by the terms you're using. When you say "observed bandwidth" do you mean the relay's 'advertised bandwidth' a relay operator puts in the config, the unit-less values the bwauths calculate and vote on, or a third thing I'm not recalling? (And if it is the third thing, where does that get exposed: Consensus, MicroConsensus, Descriptor, Microdescriptor, or ExtraInfo?)

By "observed bandwdith" I mean the "bandwdith-observed" figure from the descriptor "bandwdith" line: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

I'd like it more easily available because it's one of the two figures that an operator doesn't set themselves - the other is the measured bandwidth.

And used in conjunction with the advertised bandwidth to calculate the bandwidth in the consensus "w" line: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2268

(That figure is useful, but more confusing, because it's sometimes the advertised bandwidth, and sometimes the observed bandwidth.)

And .observed_bandwidth from the descriptor also

And then the same question for "consensus weight".

Whatever Atlas uses for consensus weight in the relay details page. I assume it's measured bandwidth from the consensus w line, but I'm not sure.

If it is, it's already there in the consensus column of the relay flags table.

https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n2274

Yes, I'm 95% sure this is working correctly currently.

Looking closer at the stem documentation, I might have done things wrong in #20372 (moved) actually. Adding Damian to confirm the below is correct:

• On a vote .measured is the bwauth's vote for the relay's (unitless) bandwidth value
• On a consensus .measured is the average* of the bwauths votes for the relay's (unitless) bandwidth value
• On a vote or consensus .bandwidth is the relay's advertised bandwidth (and is what teor wants available)

Not quite, I think it's the minimum of advertised and observed, which confuses a lot of operators, because sometimes it's controlled by the torrc, and other times by their relay's self-reported performance: https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt#n418

*not really average but let's pretend it is for simplicity

(low-median)

Ah interesting, so it just takes the median value (and if there's an even number, the lower of the two in the middle? Didn't realize that.

(Currently, the value on each Authority is the .measured but the value on the consensus is .bandwidth - which would be wrong as the consensus would be reporting the self-reported advertised bandwidth...)

That's strange, because all the figures in the table appear to be a low-median (consensus .measured) to me.

Yup, turns out the stem documentation seems to be wrong.

Trac:
Owner: N/A to tom
Component: Metrics to Metrics/Consensus Health

Opened #20493 (closed) for the stem documentation.

Adding metrics-team to cc

Trac:
Cc: tom, atagar to tom, atagar, metrics-team

mentioned in issue #20493 (closed)

mentioned in issue #20763 (moved)