Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T16:13:51Zhttps://gitlab.torproject.org/legacy/trac/-/issues/27362(sub-)packages outside of core (cli) should not need to know about confs and ...2020-06-13T16:13:51Zjuga(sub-)packages outside of core (cli) should not need to know about confs and argsThat would help to have a more modular design and use other (sub)packages and (sub)modules without the need of creating ConfigParser and ArgumentParser objects.
It also would help to simplify tests configurations.
Additionally, a progra...That would help to have a more modular design and use other (sub)packages and (sub)modules without the need of creating ConfigParser and ArgumentParser objects.
It also would help to simplify tests configurations.
Additionally, a program should take into account in this order:
- cli arguments
- environment variables
- user configuration files
- system configuration files
- program defaults
That is currently almost match. but it would be better if they all can be took into account in a simpler way.
This is not for MVP, but creating the ticket cause i'm creating new code taking this into account, and would be nice to change at some point.
Some tickets, as #27358, happen because of this.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29057Adapt bandwidth file classes to be compatible with stem (descriptors, etc) do...2020-06-13T16:15:21ZjugaAdapt bandwidth file classes to be compatible with stem (descriptors, etc) documentsso that the code can be moved to stem, as commented in #29056so that the code can be moved to stem, as commented in #29056sbws: unspecifiedjugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/28758Add requirements files2020-06-13T16:15:05ZanadahzAdd requirements filesAdd a requirements.txt file with the required sbws dependencies.Add a requirements.txt file with the required sbws dependencies.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33150Allow to connect to an external control port2020-06-13T16:16:25ZjugaAllow to connect to an external control portTo be able to run integration tests with chutney.To be able to run integration tests with chutney.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/5457Bw auths don't count circuit failures in descriptor mode2022-03-10T15:30:22ZMike PerryBw auths don't count circuit failures in descriptor modeWhen we are using descriptor bandwidth (ie no feedback), we are unable to properly use circuit failure statistics to penalize nodes that are either attempting path bias, or are just experiencing CPU overload.
The fix *should* be simple....When we are using descriptor bandwidth (ie no feedback), we are unable to properly use circuit failure statistics to penalize nodes that are either attempting path bias, or are just experiencing CPU overload.
The fix *should* be simple. I think we just need to add another clause in aggregate.py where we check for use_circ_fails to also check for use_desc_bw and properly combine the pid_error and circ_error for that case (perhaps just by multiplying them).sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/16559bwauth code needs to be smarter about failed circuits2020-06-13T16:13:26ZTvdWbwauth code needs to be smarter about failed circuitsIn the current bandwidth authority code, when a fetch attempt fails, it will still be counted as a circuit that went through all of the nodes -- even if those nodes weren't responsible for the failure.
This has the potential of resultin...In the current bandwidth authority code, when a fetch attempt fails, it will still be counted as a circuit that went through all of the nodes -- even if those nodes weren't responsible for the failure.
This has the potential of resulting in a relay not being measured sufficiently, or at all: the code will consider failures from unstable nodes to be relevant for nodes that are perfectly stable.
In slices where exits and entries aren't well-distributed (like, all of them) this can result in some nodes not being measured at all, and losing their consensus weight. This seems to affect exits a lot more than it does other relay types: people on tor-relays@ have mentioned that removing their exit policies gets their consensus weight back, and I have been able to reproduce this.sbws: unspecifiedjugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/25925bwauth improvements (ex-parent ticket for SoP planned tasks)2020-06-13T16:13:27Zjugabwauth improvements (ex-parent ticket for SoP planned tasks)sbws: unspecifiedjugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/2550bwauth should reschedule quicker bandwidth test when bandwidthrate changes?2022-02-17T19:36:19ZRoger Dingledinebwauth should reschedule quicker bandwidth test when bandwidthrate changes?https://metrics.torproject.org/relay-search.html?search=AEIOUm+2011-02-13
He apparently switches between 100KB/s and 2MB/s bandwidthrate depending on time of day. His bwauth votes ended up being very skewed:
```
moria1 says
w Bandwidth...https://metrics.torproject.org/relay-search.html?search=AEIOUm+2011-02-13
He apparently switches between 100KB/s and 2MB/s bandwidthrate depending on time of day. His bwauth votes ended up being very skewed:
```
moria1 says
w Bandwidth=141 Measured=15
ides says
w Bandwidth=141 Measured=26
urras says
w Bandwidth=141 Measured=1480
gabelmoo says
w Bandwidth=141 Measured=935
```
It's a shame that we're giving really low numbers to a node that wants to be 2MB/s at some times of day. It probably also means we give high numbers to a slow node if the measurement times are different.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29851Catch SIGHUP to be able to reload configuration without stopping the scanner2020-06-13T16:15:46ZjugaCatch SIGHUP to be able to reload configuration without stopping the scannersbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/28106Change integration tests from bash to shell2020-06-13T16:14:12ZjugaChange integration tests from bash to shellgman999 reported that tests need to change from bash to shell to include them in openbsdgman999 reported that tests need to change from bash to shell to include them in openbsdsbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29716Change the way the consensus is received2020-06-13T16:15:35ZjugaChange the way the consensus is receivedCurrently it receives the new consensuses using https://stem.torproject.org/api/control.html#stem.control.Controller.get_network_statuses, which uses the cached-consensus and might not be updated.
It should probably receive the new conse...Currently it receives the new consensuses using https://stem.torproject.org/api/control.html#stem.control.Controller.get_network_statuses, which uses the cached-consensus and might not be updated.
It should probably receive the new consensuses as soon as they arrive using an event listener https://stem.torproject.org/api/control.html#stem.control.Controller.add_event_listener, instead of every x time.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29727Check output of generate in the integration tests2020-06-13T16:15:41ZjugaCheck output of generate in the integration testsIt was included `sbws scanner` in the tests, but `generate` could also be included and check that the files generated contain expectected Keys and Values.
These files could also replace many of the tests input examples and change them a...It was included `sbws scanner` in the tests, but `generate` could also be included and check that the files generated contain expectected Keys and Values.
These files could also replace many of the tests input examples and change them as part of #28684sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29783Consider measuring all the network at once2020-06-13T16:15:45ZjugaConsider measuring all the network at onceCurrently sbws has the "prioritization loop" and measures a subsets of relays.
In #28547 we try to monitor all the relays that does not get measured.
It would be probably way simpler to just measure all the relays currently in the networ...Currently sbws has the "prioritization loop" and measures a subsets of relays.
In #28547 we try to monitor all the relays that does not get measured.
It would be probably way simpler to just measure all the relays currently in the network, which will tell us in one loop and around 30 hours whether not all relays were measured.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29582Create a git hook to set default commit message that will help automate releases2020-06-13T16:15:30ZjugaCreate a git hook to set default commit message that will help automate releasesCommented in #29294 revision.Commented in #29294 revision.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/33152Create a tool to detect issues in the bandwidth files given their key/values2020-06-13T16:16:25ZjugaCreate a tool to detect issues in the bandwidth files given their key/valuesIt would be nice to have a tool that reports issues based on the bandwidth files key/values.
For example, if a relay has been seen in less consensuses that it's known it was.
Or to report issues similar to the ones mentioned in #29954 ch...It would be nice to have a tool that reports issues based on the bandwidth files key/values.
For example, if a relay has been seen in less consensuses that it's known it was.
Or to report issues similar to the ones mentioned in #29954 child tickets.
This tool could contain #30735sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29294Create an script to automate releases2020-06-13T16:15:26ZjugaCreate an script to automate releasessbws: unspecifiedjugajugahttps://gitlab.torproject.org/legacy/trac/-/issues/29753Create an script to detect bugs with the KeyValues to monitor not reported re...2020-06-13T16:15:43ZjugaCreate an script to detect bugs with the KeyValues to monitor not reported releayAfter #28547 is solved, there will be KeyValues in the bandwidth file that can help to find bugs in sbws.
And after #21377 is solved, anyone will be able to see the bandwidth files.
There should be an script that can automatically detect...After #28547 is solved, there will be KeyValues in the bandwidth file that can help to find bugs in sbws.
And after #21377 is solved, anyone will be able to see the bandwidth files.
There should be an script that can automatically detect these bugs.
This is partly documented in https://pad.riseup.net/p/sbws-exclude-reasons-keep.
The script should probably be in other package/component so that anyone can run it.
Assigning for now to `sbws` component since depends on `sbws`.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/10791Detect overtuned exit relays2020-06-13T16:13:25ZcypherpunksDetect overtuned exit relaysMany relays breaks usability for users by trying so small time of syn packet that connection timeout is very low if relay overloaded and syn was lost. Tor client have no ability to retry with another circuit if reason for end cell is tim...Many relays breaks usability for users by trying so small time of syn packet that connection timeout is very low if relay overloaded and syn was lost. Tor client have no ability to retry with another circuit if reason for end cell is timeout or refused. By default Tor can to retry only if no answer for 10 (15) seconds or some end reasons.
Torflow should to test exit relays if they answers timeout faster than after 15 seconds or refuses for known working host.
Relays that overtuned or overfirewalled still usefull as non-exit relays but should be marked as BadExits if stable Tor client version can't to retry.sbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/27343Dockerfile for sbws basic install2020-06-13T16:13:48ZTracDockerfile for sbws basic install
```
FROM debian
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -qy install python3-pip git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN useradd --shell /bin/bash -u 1000...
```
FROM debian
ENV DEBIAN_FRONTEND=noninteractive
RUN apt-get update && apt-get -qy install python3-pip git && \
apt-get clean && \
rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/*
RUN useradd --shell /bin/bash -u 1000 -m test
USER test
RUN git clone https://git.torproject.org/stem.git /home/test/stem
RUN git clone https://github.com/pastly/simple-bw-scanner.git /home/test/simple-bw-scanner
USER root
RUN pip3 install /home/test/stem
RUN pip3 install /home/test/simple-bw-scanner
USER test
CMD /usr/local/bin/sbws scanner
```
Just a basic Dockerfile to get sbws installed inside Docker. Note that it needs to be adapted a little bit, as it lacks a config file... Whenever I figure out the config file aspect, I can get that included easily.
**Trac**:
**Username**: gabesbws: unspecifiedhttps://gitlab.torproject.org/legacy/trac/-/issues/29752Document new sbws trac keywords2020-06-13T16:15:43ZjugaDocument new sbws trac keywordssbws: unspecified