Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T16:16:42Zhttps://gitlab.torproject.org/legacy/trac/-/issues/34310Maybe add new type of errors in the bandwidth file2020-06-13T16:16:42ZjugaMaybe add new type of errors in the bandwidth fileAs commented in https://trac.torproject.org/projects/tor/ticket/30905#comment:9
Things that are not really failures:
when loading the results, the relays that changed ip are excluded. I could add a patch to count all the results fi...As commented in https://trac.torproject.org/projects/tor/ticket/30905#comment:9
Things that are not really failures:
when loading the results, the relays that changed ip are excluded. I could add a patch to count all the results first, which reduces the number of failures in around 1000.
when sbws is restarted, all the queued relays to measure that didn't start to be measured yet, are not saved in the results, but the attempt has already been count. We could also add a patch to count the attempt in measure_relay, instead of main_loop, though then we won't know the number of attempts counting from the moment they were queued.
real failures would happen when result_putter_error is triggered, or some bug makes sbws stall, which then will print the TICKET_MSG error. In the 1st case, we could actually count an error.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33590SBWS maatuska is broken2020-06-13T16:16:32ZstarlightSBWS maatuska is brokenhttps://metrics.torproject.org/totalcw.png?start=2019-12-18
```
tingpo
bastet--- 3600 0.0150 +0.4605 0 39
Faravahar 4980 0.0208 +1.0203 0 39
maatuska- 2275652 9.4867 +922.1891 0 39
NLRe...https://metrics.torproject.org/totalcw.png?start=2019-12-18
```
tingpo
bastet--- 3600 0.0150 +0.4605 0 39
Faravahar 4980 0.0208 +1.0203 0 39
maatuska- 2275652 9.4867 +922.1891 0 39
NLRelay05
bastet--- 1 0.0000 -0.9998 34 1747
moria1--- 5770 0.0239 +0.1819 34 1747
maatuska- 1870361 7.7590 +382.1240 34 1747
```sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33571Rename constants containing list of keys in v3bwfile.py2020-06-13T16:16:31ZjugaRename constants containing list of keys in v3bwfile.pyOr refactor code to do not have to maintain all those lists.
From comment in https://trac.torproject.org/projects/tor/ticket/30196#comment:21Or refactor code to do not have to maintain all those lists.
From comment in https://trac.torproject.org/projects/tor/ticket/30196#comment:21sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33473Document which branches developers should base their patches on2020-06-13T16:16:30ZjugaDocument which branches developers should base their patches onsbws: unspecifiedjugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/33197Refactor code that obtain headers from the state file2020-06-13T16:16:27ZjugaRefactor code that obtain headers from the state fileWorking on #30196 there was a commit to do what the title says. It was suggested to do it in a different PR.Working on #30196 there was a commit to do what the title says. It was suggested to do it in a different PR.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33167Test changes in descriptors2020-06-13T16:16:26ZjugaTest changes in descriptorsWorking on #30733 asn pointed out that there're no tests to check the changes in descriptors.
It'll be easier to create these tests when:
- sbws can use chutney (#33150) so that the test network lives longer than the 1st consensus
- `R...Working on #30733 asn pointed out that there're no tests to check the changes in descriptors.
It'll be easier to create these tests when:
- sbws can use chutney (#33150) so that the test network lives longer than the 1st consensus
- `RelayList` can be initialized with different consensus/descriptors and not only a controller (#29717)
- have an external tool that check the generated bandwidth files in the Tor network and detects changes in the descriptors (#33152)
Maybe we should include this ticket as part of #33121sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33166Test tor dormant mode2020-06-13T16:16:26ZjugaTest tor dormant modeWorking on #30733 asn pointed out to check the dormant mode.
Dormant mode was introduced in tor 0.4.0.4-rc.
As teor suggested, we should check whether sbws stop trying to make circuits, and therefore become dormant, when:
- the network...Working on #30733 asn pointed out to check the dormant mode.
Dormant mode was introduced in tor 0.4.0.4-rc.
As teor suggested, we should check whether sbws stop trying to make circuits, and therefore become dormant, when:
- the network goes down (it probably just stops and prints a traceback, as happened in #32939)
- tor doesn't have a recent consensus or descriptors
We also should check what happen when sbws starts with tor in dormant mode and probably set `DormantCanceledByStartup 1` until we fix bugs.
Maybe we should include this ticket as part of #33121.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33152Create a tool to detect issues in the bandwidth files given their key/values2020-06-13T16:16:25ZjugaCreate a tool to detect issues in the bandwidth files given their key/valuesIt would be nice to have a tool that reports issues based on the bandwidth files key/values.
For example, if a relay has been seen in less consensuses that it's known it was.
Or to report issues similar to the ones mentioned in #29954 ch...It would be nice to have a tool that reports issues based on the bandwidth files key/values.
For example, if a relay has been seen in less consensuses that it's known it was.
Or to report issues similar to the ones mentioned in #29954 child tickets.
This tool could contain #30735sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33150Allow to connect to an external control port2020-06-13T16:16:25ZjugaAllow to connect to an external control portTo be able to run integration tests with chutney.To be able to run integration tests with chutney.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33149Parent ticket to improve test coverage and integration tests2020-06-13T16:16:24ZjugaParent ticket to improve test coverage and integration testsParent ticketParent ticketsbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/31331scanner integration test fails some times because the test Web server is not ...2020-06-13T16:16:18Zjugascanner integration test fails some times because the test Web server is not multi-threadedBecause the Python Web server used for the integration tests is not multi-threaded, some relays' circuits timeout.
Most of the cases, this is desired since in a public network this would happen too.
It is not desired in the case that the...Because the Python Web server used for the integration tests is not multi-threaded, some relays' circuits timeout.
Most of the cases, this is desired since in a public network this would happen too.
It is not desired in the case that the integration test check for a concrete relay measurement success, which is `tests/integration/core/test_scanner.py::test_measure_relay_with_maxadvertisedbandwidth` (https://travis-ci.org/torproject/sbws/jobs/565274602#L1971).
This case can be solved by checking the descriptor bandwidth, not the measurement.
Ideally, the Python Web server should be changed to be multi-threaded and shutting down some relays for the cases where the tests check failures.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/30907Write the contents of the state file in order2020-06-13T16:16:16ZjugaWrite the contents of the state file in orderSince then it'll be deterministid what is useful to have integration tests that check the state file.Since then it'll be deterministid what is useful to have integration tests that check the state file.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/30902Stop comparing file_created key in test2020-06-13T16:16:13ZjugaStop comparing file_created key in testIn concrete tests/unit/lib/v3bwfile.py::test_from_results_read.
Because some times it files with the second: https://travis-ci.org/juga0/sbws/jobs/546378459#L1339.In concrete tests/unit/lib/v3bwfile.py::test_from_results_read.
Because some times it files with the second: https://travis-ci.org/juga0/sbws/jobs/546378459#L1339.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29954Parent ticket for scanner improvements2022-04-26T07:58:37ZjugaParent ticket for scanner improvementssbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29953Parent ticket for easy tickets that do not change version2020-06-13T16:15:49ZjugaParent ticket for easy tickets that do not change versionsbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29892Easy deployments in test server running in public network2020-06-13T16:15:48ZjugaEasy deployments in test server running in public networkSome bugs can only be detected after running the scanner several hours in the public network.
Having a fabfile or similar would easir that deploying automatically in a test server.Some bugs can only be detected after running the scanner several hours in the public network.
Having a fabfile or similar would easir that deploying automatically in a test server.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29851Catch SIGHUP to be able to reload configuration without stopping the scanner2020-06-13T16:15:46ZjugaCatch SIGHUP to be able to reload configuration without stopping the scannersbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29838Update trac wiki pages where sbws should be listed2020-06-13T16:15:46ZjugaUpdate trac wiki pages where sbws should be listedAll the pages where it's listed: https://trac.torproject.org/projects/tor/search?q=torflow&noquickjump=1&wiki=on
For instance, include sbws in https://trac.torproject.org/projects/tor/wiki/org/projects
Remove torflow when bandwidth auth...All the pages where it's listed: https://trac.torproject.org/projects/tor/search?q=torflow&noquickjump=1&wiki=on
For instance, include sbws in https://trac.torproject.org/projects/tor/wiki/org/projects
Remove torflow when bandwidth authorities are happy with sbws.
Adding this in sbws as there's no component for trac pages.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29783Consider measuring all the network at once2020-06-13T16:15:45ZjugaConsider measuring all the network at onceCurrently sbws has the "prioritization loop" and measures a subsets of relays.
In #28547 we try to monitor all the relays that does not get measured.
It would be probably way simpler to just measure all the relays currently in the networ...Currently sbws has the "prioritization loop" and measures a subsets of relays.
In #28547 we try to monitor all the relays that does not get measured.
It would be probably way simpler to just measure all the relays currently in the network, which will tell us in one loop and around 30 hours whether not all relays were measured.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29753Create an script to detect bugs with the KeyValues to monitor not reported re...2020-06-13T16:15:43ZjugaCreate an script to detect bugs with the KeyValues to monitor not reported releayAfter #28547 is solved, there will be KeyValues in the bandwidth file that can help to find bugs in sbws.
And after #21377 is solved, anyone will be able to see the bandwidth files.
There should be an script that can automatically detect...After #28547 is solved, there will be KeyValues in the bandwidth file that can help to find bugs in sbws.
And after #21377 is solved, anyone will be able to see the bandwidth files.
There should be an script that can automatically detect these bugs.
This is partly documented in https://pad.riseup.net/p/sbws-exclude-reasons-keep.
The script should probably be in other package/component so that anyone can run it.
Assigning for now to `sbws` component since depends on `sbws`.sbws: unspecified