Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2022-04-26T07:58:38Zhttps://gitlab.torproject.org/legacy/trac/-/issues/30735Work out which relays are ignored by all sbws instances2022-04-26T07:58:38ZteorWork out which relays are ignored by all sbws instancesWe need to find out if sbws is excluding any relays.
If the number of excluded relays is small, we can go below 3 torflow instances.We need to find out if sbws is excluding any relays.
If the number of excluded relays is small, we can go below 3 torflow instances.sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/28983Work out how long it takes sbws to measure the network2022-02-17T17:35:05ZteorWork out how long it takes sbws to measure the networkIn the bandwidth file, sbws should report how long it takes to measure (almost all) the entire network.
Here's one way we could work out the network measurement "half-life":
1. Count the number of relays in the network, N
2. Find the la...In the bandwidth file, sbws should report how long it takes to measure (almost all) the entire network.
Here's one way we could work out the network measurement "half-life":
1. Count the number of relays in the network, N
2. Find the last N/2 successful, unique relay measurements
3. Report how much time has elapsed since the earliest measurement in the list
(We use the half-life, so that failed measurements don't affect the final time too much.)
We could monitor this figure for each scanner, to make sure they are working correctly.sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/33121Deploy sbws in the rest of bwauths2022-02-17T10:44:00ZjugaDeploy sbws in the rest of bwauthsWhen we're ready to deploy sbws in all the remaining bwauths (3/6), send an email to the dirauths to start the transition process.When we're ready to deploy sbws in all the remaining bwauths (3/6), send an email to the dirauths to start the transition process.sbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/30733sbws does not detect changes in descriptor bandwidth values2022-02-17T10:43:59Zstarlightsbws does not detect changes in descriptor bandwidth valuesSee attached files. Relay powerlay lowered max advertised bandwidth late 5/23. SBWS longclaw never recognized the change. SBWS bastet saw the value but was started after the change. Torflow recognized the change instantly.
~~ Have o...See attached files. Relay powerlay lowered max advertised bandwidth late 5/23. SBWS longclaw never recognized the change. SBWS bastet saw the value but was started after the change. Torflow recognized the change instantly.
~~ Have observed that SBWS, when it makes note of max advertised bandwidth, averages the value. This value should not be averaged, the current value whatever it is apples. ~~sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/33350Is sbws weighting some relays too high?2022-02-07T19:22:53ZteorIs sbws weighting some relays too high?Before we deploy sbws to the rest of the bandwidth authorities, we should check if it is weighting some relays (or some ASes) much higher than torflow.
We should also check for bugs in sbws, that weight existing large ASes too high:
htt...Before we deploy sbws to the rest of the bandwidth authorities, we should check if it is weighting some relays (or some ASes) much higher than torflow.
We should also check for bugs in sbws, that weight existing large ASes too high:
https://metrics.torproject.org/rs.html#aggregate/assbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/34309Check that relay_recent_measurement_attempt_count and relay_recent_priority_l...2021-03-09T16:28:13ZjugaCheck that relay_recent_measurement_attempt_count and relay_recent_priority_list_count are correcThis was a comment in https://trac.torproject.org/projects/tor/ticket/30905#comment:9This was a comment in https://trac.torproject.org/projects/tor/ticket/30905#comment:9sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/30905Maybe monitoring values in the state file should be reset when sbws is restarted2021-03-03T16:49:49ZjugaMaybe monitoring values in the state file should be reset when sbws is restartedI think the following keys should probably be restarted, to make less confusing what appears in the bandwidth file. For instance, after restarting longclaw's sbws, this is what the state file contains, consensus count and priority lists ...I think the following keys should probably be restarted, to make less confusing what appears in the bandwidth file. For instance, after restarting longclaw's sbws, this is what the state file contains, consensus count and priority lists are reset, but not the others:
```
"recent_measurement_attempt_count": 671,
"recent_consensus_count": 1,
"recent_priority_relay_count": 984,
"recent_priority_list_count": 3,
"min_perc_reached": "2019-05-06T06:20:49",
```sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/34394Test net keys expired2020-06-13T16:16:43ZjugaTest net keys expiredAnd the integration tests fail cause test net dirauths don't generate consensuses. Travis error https://travis-ci.org/github/torproject/sbws/jobs/693690384, though that doesn't show dirauths tor log.
One more problem that could be avoide...And the integration tests fail cause test net dirauths don't generate consensuses. Travis error https://travis-ci.org/github/torproject/sbws/jobs/693690384, though that doesn't show dirauths tor log.
One more problem that could be avoided implementing #33150.sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/34393Maybe vote on the relays that have few or close measurements2020-06-13T16:16:42ZjugaMaybe vote on the relays that have few or close measurementsTo vote on aprox. the same number of relays as Torflow.
Toflow does not do it.To vote on aprox. the same number of relays as Torflow.
Toflow does not do it.sbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33947Compare sbws and Torflow2020-06-13T16:16:41ZjugaCompare sbws and Torflowgk and i were talking about what to review in #30375 and we thought it'd be useful to create a ticket to check whether the bugfixes we have been working on (https://trac.torproject.org/projects/tor/query?keywords=~sbws-roadmap&status=clo...gk and i were talking about what to review in #30375 and we thought it'd be useful to create a ticket to check whether the bugfixes we have been working on (https://trac.torproject.org/projects/tor/query?keywords=~sbws-roadmap&status=closed) to deploy sbws in all bwauths are working, ie. making sbws to behave very close to Torflow.
I think we should document what to check, where/how to check it and which ticket(s) intended to fix it.
I also think we should add this as documentation in sbws itself because they're important questions that have been blockers to deploy sbws in all bwauths and to avoid regressions in the future.
Some of the main things to check that should be further explained are:
- whether sbws "failures" "low" (#30719)
- whether the number of relays to vote on reported by sbws "similar" to the number of relays reported by torflow (#30727, #30735)
- whether sbws relay descriptors are updated (#30733)
- whether sbws router statuses (relay info. from consensus) are updated (#30733)
- whether sbws consensus bandwidth total sum is similar to torflow (#33871, #33009, #33350)?
- whether changes in a relay consensus bandwidth affect in a similar way as torflow (#33871)sbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33871Scale exactly as torflow does?2020-06-13T16:16:40ZjugaScale exactly as torflow does?#33775 shows that if sbws calculates low consensus bandwidth because of missing descriptors and the rest of bwauths are also using sbws, it'd enter in a spiral in which it'd keep measuring low.
I think this is because while torflow multi...#33775 shows that if sbws calculates low consensus bandwidth because of missing descriptors and the rest of bwauths are also using sbws, it'd enter in a spiral in which it'd keep measuring low.
I think this is because while torflow multiplies the calculated ratio by the descriptor observed bandwidth [0], while sbws multiplies the ratio by the minimum of all descriptor bandwidth values *and* the consensus, which was added in #28598.
So maybe the new consensus bandwidth should not depend on the previous one, or not as the minimum.
For a relation on how bandwidth values depend on each other, see [1]
[0] https://onbasca.readthedocs.io/en/latest/torflow_aggr.html
[1] https://onbasca.readthedocs.io/en/latest/bandwidth_tor.html#bandwidth-values-originsbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/33832For relays that change ip, only the measurements with the last ip are kept2020-06-13T16:16:38ZjugaFor relays that change ip, only the measurements with the last ip are keptwhich makes those relays more likely to don't be "eligible" cause don't have enough results and therefore, sbws voting in less relays.which makes those relays more likely to don't be "eligible" cause don't have enough results and therefore, sbws voting in less relays.sbws: 1.1.x-finalGeorg KoppenGeorg Koppenhttps://gitlab.torproject.org/legacy/trac/-/issues/33831Relays without descriptors are not scaled, but still added to the bwlines wit...2020-06-13T16:16:37ZjugaRelays without descriptors are not scaled, but still added to the bwlines without vote=0As can be seen in: https://gitweb.torproject.org/sbws.git/tree/sbws/lib/v3bwfile.py?h=maint-1.1#n1317
As a result, some relays (in sample data counted ~800) are included in the bandwidth file without its bandwidth scaled, which could b...As can be seen in: https://gitweb.torproject.org/sbws.git/tree/sbws/lib/v3bwfile.py?h=maint-1.1#n1317
As a result, some relays (in sample data counted ~800) are included in the bandwidth file without its bandwidth scaled, which could be quite different (higher or lower) than the scaled bandwidth.
This is one of the several reasons of #33775.sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/33775sbws measures some relays 100x lower than Torflow2020-06-13T16:16:35Zteorsbws measures some relays 100x lower than TorflowHi,
I've received multiple reports from relay operators who are concerned their relays are being measured too low.
At the moment, we only have 2 Torflow instances in the network, and 3 sbws instances. So bad sbws measurements can lower...Hi,
I've received multiple reports from relay operators who are concerned their relays are being measured too low.
At the moment, we only have 2 Torflow instances in the network, and 3 sbws instances. So bad sbws measurements can lower any relay's bandwidth, regardless of the Torflow measurement.
Here is one report:
https://lists.torproject.org/pipermail/tor-relays/2020-March/018321.html
At the moment, the bandwidths are:
moria1 494
maatuska 4
longclaw 5
https://consensus-health.torproject.org/consensus-health-2020-03-31-21-00.html#6E1DA4C0B0C05FB721B42329C47A20DA22908AEB
I have also received a similar report privately from a large relay operator.
This bug could be related to #33009 or #30733.sbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33691Migrate to gitlab.tpo2020-06-13T16:16:34ZjugaMigrate to gitlab.tpoSteps to start doing reviews of sbws to gitlab.tpo:
- create sbws repository that will be used by the MR, either in the gitlab root or network healt team
- clone our personal repositories from the previous one
- create a .gitlab-ci.yml ...Steps to start doing reviews of sbws to gitlab.tpo:
- create sbws repository that will be used by the MR, either in the gitlab root or network healt team
- clone our personal repositories from the previous one
- create a .gitlab-ci.yml similar to .travis.yml, so that the sbws can be automatically tested
Steps to migrate completely to gitlab.tpo:
- migrate Trac tickets
- migrate Trac wiki page about sbws
- migrate GitHub issuessbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33570Correct the relays to keep after retrieving new consensuses2020-06-13T16:16:31ZjugaCorrect the relays to keep after retrieving new consensusesWhen removing the old consensus timestamps, the oldest date is calculated with timedelta without passing seconds, which was taking it as days, making the oldest date very far in the past.
Recent timestamps are calculated as greater than ...When removing the old consensus timestamps, the oldest date is calculated with timedelta without passing seconds, which was taking it as days, making the oldest date very far in the past.
Recent timestamps are calculated as greater than the oldest date, instead of minor, what would make to store many timestamps.
The most recent timestamp for a relay, was not being took into account, because the router status was being updated after updating the timestamps.sbws: 1.1.x-finaljugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/33472Document that bwauths should checkout stable versions when installing sbws fr...2020-06-13T16:16:30ZjugaDocument that bwauths should checkout stable versions when installing sbws from giti think that if some bwauths are installing sbws from git, they should checkout a tag or a bugfix branch. We might want to have a bwauth to run a development branch, but it should be only one.i think that if some bwauths are installing sbws from git, they should checkout a tag or a bugfix branch. We might want to have a bwauth to run a development branch, but it should be only one.sbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33199Fix lint error after merging #307332020-06-13T16:16:28ZjugaFix lint error after merging #30733Because there was a merge conflict merging #30733 after merging #30727, we accepted changes from both, but a new line is missing and flake8 complains (https://travis-ci.org/juga0/sbws/jobs/648108391#L2024)Because there was a merge conflict merging #30733 after merging #30727, we accepted changes from both, but a new line is missing and flake8 complains (https://travis-ci.org/juga0/sbws/jobs/648108391#L2024)sbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33198Check changes related to descriptors in a bandwidth file created by a bwauth ...2020-06-13T16:16:28ZjugaCheck changes related to descriptors in a bandwidth file created by a bwauth before next releaseIn #30733 we did not create tests for the changes updating descriptors nor dormant mode.
Until those tests are implemented, we could change a bwauth to run the latest maint-1.1 branch and look the bandwidth files produced.
This way we co...In #30733 we did not create tests for the changes updating descriptors nor dormant mode.
Until those tests are implemented, we could change a bwauth to run the latest maint-1.1 branch and look the bandwidth files produced.
This way we could avoid some bugs before releasing a new version.
Maybe #30899 should be child of this to know that the bwauth is not running a released version but a git version.sbws: 1.1.x-finalhttps://gitlab.torproject.org/legacy/trac/-/issues/33060Check that consensus timestamp lists is initialied and has at list one item2020-06-13T16:16:22ZjugaCheck that consensus timestamp lists is initialied and has at list one itemWorking on #30909 realized that even if I realized that when checking ` if len(self._consensus_timestamps) >= 1`, in the case that `self._consensus_timestamps` wouldn't not be initialized, it'd give a type error.
That doesn't currently h...Working on #30909 realized that even if I realized that when checking ` if len(self._consensus_timestamps) >= 1`, in the case that `self._consensus_timestamps` wouldn't not be initialized, it'd give a type error.
That doesn't currently happen because that code is call after the `__init__` method. It'd still be safer to to check that it's been initialized.sbws: 1.1.x-final