Chutney generates network with no bandwidth weights

added SponsorS component::core tor/chutney priority::medium severity::normal status::needs-information type::defect labels

Hi asn,

Setting these flags on every relays was a part of the rapid bootstrap changes that I made in #13163 (moved) and related issues.

You're right, it would be great to be able to select parts of the network for these flags. I can imagine this feature being implemented like this:

TestingDirAuthVoteExit m-n
TestingDirAuthVoteGuard p%-q%

This would vote relays m to n (by fingerprint) as Exits, and relays p% to q% as Guards. We'd probably want to make a similar change to the corresponding TestingDirAuthVoteHSDir flag as well, even if we didn't use it.

We could implement this feature in chutney or in tor:

In chutney, we could perhaps do this as part of the templating system, or maybe it could wait for chutney 2?
In tor, we could allow the TestingDirAuthVote... parameters to take a m-n or p%-q% argument, by modifying routerset_contains_routerstatus. I'd want to ensure this was only active in test networks, of course.

I am in favour of making this change in tor, where it might assist a wider range of users. And it would work in chutney 2 for free. But I'd want to get nickm's advice.

What do you think?

Replying to teor:

We could implement this feature in chutney or in tor:

In chutney, we could perhaps do this as part of the templating system, or maybe it could wait for chutney 2?

In tor, we could allow the TestingDirAuthVote... parameters to take a m-n or p%-q% argument, by modifying routerset_contains_routerstatus. I'd want to ensure this was only active in test networks, of course.

I am in favour of making this change in tor, where it might assist a wider range of users. And it would work in chutney 2 for free. But I'd want to get nickm's advice.

Hehe. My initial intuition would be to build this feature in chutney.

I prefer our intrusions to the Tor codebase for the purposes of testing networks to be minimal, if it can be helped. Also, chutney is in Python and I imagine that editing the templating logic of chutney shouldn't be too hard for this purpose.

However, if there is indeed a wider range of users, then we should think about it more. Who is doing testing networks with something other than chutney and why? Maybe Shadow needs this feature, but I'm not sure.

I agree - I'd prefer testing infrastructure to be external as much as possible. Makes it easier to deploy and change.

However, this feature could be useful for Shadow, and various people on tor-dev and tor-relays who seem to enjoy setting up test networks without using TestingTorNetwork or chutney. It's almost a requirement to use TestingDirAuthVote... if you want a Tor network to bootstrap in a few seconds rather than minutes.

I'd hate to exclude guardfraction and similar features from testing, just because we don;t have sufficient support in the tor codebase to make them work in test networks. (Does that make sense?)

I'd be happy to prototype it in chutney and see if that's sufficient, or ask on tor-dev - I don't know, what do you think?

I think the second issue here, is that by default chutney relays will publish zero bandwidth. That's because there are no bwauths, and they also don't see any traffic so their bandwidth self-evaluation fails.

My dirty fix is to make each relay publish a random value as its self-reported bandwidth, but it's a bad fix.

Have you tried using CHUTNEY_DATA_BYTES=104857600 chutney verify? (That command sends 100MB through each client and HS. It was merged to chutney in #14175 (moved).)

Although, I wonder how often the self-reported bandwidths are reported. Maybe we need to add them to #13976 (moved) to get fixed up in test networks.

This ticket may also be related to #14034 (moved) (which is about making test networks behave more like the real thing).

Trac:
Parent: N/A to #13976 (moved)

Still trying to work out the best approach to this.

Trac:
Keywords: N/A deleted, SponsorS added
Status: new to needs_information

I think we need to reduce CHECK_DESCRIPTOR_INTERVAL and perhaps BANDWIDTH_RECHECK_INTERVAL in test networks. They should probably get a Testing* option and be defaulted to their current values (1 minute, 12 hours); but be ~4 seconds and ~16 seconds by default (and configurable) in test networks.

This will allow us to:

Use TestingDirAuthVoteExit, TestingDirAuthVoteGuard, and TestingDirAuthVoteGuardIsStrict to set up the network (#14882 (moved), git 359faf5e4b4a88663201c2b42dd89a6f77569856)
Run CHUTNEY_DATA_BYTES=104857600 ./chutney verify to add some data to the network (this step might be able to be skipped if self-testing adds enough data by itself)
Wait a few seconds for the values to be included in the consensus

Trac:
Cc: N/A to ln5

This change has a chutney component and a tor component:

chutney: automatically assign 25% guards, exits, guards + exits, middles (asn, are there any other categories we need here?) - this task
tor: report bandwidth more frequently - split into #17036 (moved)

Replying to teor:

This change has a chutney component and a tor component:

chutney: automatically assign 25% guards, exits, guards + exits, middles (asn, are there any other categories we need here?) - this task

tor: report bandwidth more frequently - split into #17036 (moved)

Sounds reasonable!

Remove myself as owner from old chutney tickets; that's not what we use owner for.

Trac:
Status: needs_information to assigned
Owner: nickm to N/A

Trac:
Status: assigned to needs_information

This was partially fixed in 0.3.0 with a new consensus method that initialises weights to 1 (see #14881 (moved)).

Trac:
Reviewer: N/A to N/A
Sponsor: N/A to N/A
Severity: N/A to Normal

We should only implement early bandwidth reporting for test networks, or networks with no -default authority sets. Implementing in the real network can expose client guards (#23856 (moved)), but if we only do fast reporting in the first 24 hours, it might be ok (#24104 (moved)).