Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T16:19:52Zhttps://gitlab.torproject.org/legacy/trac/-/issues/24221gitignore more things in torflow2020-06-13T16:19:52Zteorgitignore more things in torflowTom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24219Avoid aggregate.py error when no nodes are measured2020-06-13T16:19:51ZteorAvoid aggregate.py error when no nodes are measuredIf the scanner cron runs early, errors like this can happen:
```
Traceback (most recent call last):
File "/Users/bwauth/torflow/NetworkScanners/BwAuthority/aggregate.py", line 894, in <module>
main(sys.argv)
File "/Users/bwauth/t...If the scanner cron runs early, errors like this can happen:
```
Traceback (most recent call last):
File "/Users/bwauth/torflow/NetworkScanners/BwAuthority/aggregate.py", line 894, in <module>
main(sys.argv)
File "/Users/bwauth/torflow/NetworkScanners/BwAuthority/aggregate.py", line 798, in main
nodes.itervalues())))
ValueError: min() arg is an empty sequence
```https://gitlab.torproject.org/legacy/trac/-/issues/24216Make p global in the sigterm handler2020-06-13T16:19:51ZteorMake p global in the sigterm handlerThis causes errors like:
```
./bwauthority.py:71: SyntaxWarning: name 'p' is assigned to before global declaration
global p
```
in some recent python versions.
It appears to be one of those lovely python hiesenbugs that appears and di...This causes errors like:
```
./bwauthority.py:71: SyntaxWarning: name 'p' is assigned to before global declaration
global p
```
in some recent python versions.
It appears to be one of those lovely python hiesenbugs that appears and disappears depending on python version, OS, and other code changes.Tom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24094measured bw percentage is always 1002020-06-13T16:19:50ZSebastian Hahnmeasured bw percentage is always 100Using tor 0.3.1.7, I get this right from the first start:
NOTICE[Tue Oct 31 12:45:15 2017]:Measured 100.0% of all tor nodes (100.0% of previous consensus bw).
the bwscan-file is produced but only contains 400 lines.Using tor 0.3.1.7, I get this right from the first start:
NOTICE[Tue Oct 31 12:45:15 2017]:Measured 100.0% of all tor nodes (100.0% of previous consensus bw).
the bwscan-file is produced but only contains 400 lines.Tom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24091Recommend system-installed tor2020-06-13T16:19:50ZSebastian HahnRecommend system-installed torPointing the bwauth at a system-installed tor should not cause any trouble, I've been running the older code with that for years and the new code seems to work too. Just set TOR_EXE to the absolute path of the system tor.Pointing the bwauth at a system-installed tor should not cause any trouble, I've been running the older code with that for years and the new code seems to work too. Just set TOR_EXE to the absolute path of the system tor.Tom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24090Remove default bwserver2020-06-13T16:19:49ZSebastian HahnRemove default bwserverBy not providing a bwserver in the config, we can make sure people actually think about what they want to use. That's much better than them accidentally all using the same one.By not providing a bwserver in the config, we can make sure people actually think about what they want to use. That's much better than them accidentally all using the same one.Tom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24089requirements.txt uses peep format2020-06-13T16:19:49ZSebastian Hahnrequirements.txt uses peep formatpeep is deprecated and shouldn't be used if at least pip 8 is available (even stretch has pip 9 already). Also, it sets --no-index which prevents it from working and --no-use-wheel which is unnecessary when using pip.
This could be the ...peep is deprecated and shouldn't be used if at least pip 8 is available (even stretch has pip 9 already). Also, it sets --no-index which prevents it from working and --no-use-wheel which is unnecessary when using pip.
This could be the file:
```
pysqlite==2.6.3 --hash=sha256:fe9c35216bf56c858b34c4b4c8be7e34566ddef29670e5a5b43f9cb8ecfbb28d
SQLAlchemy==0.7.10 --hash=sha256:77aa39d65c9d043eba6ba329b359ff867424fd6c403b7c0cb112b65e507e1d66
Elixir==0.7.1 --hash=sha256:a7ef437f25b544e4f74fb3236fc43cd25f5d6feb6037dd7c66931046d75439e9
```Tom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24088Remove setup.sh2020-06-13T16:19:48ZSebastian HahnRemove setup.shSetting up a real bwauth shouldn't hide steps from the operator. It's important the operator knows what software plays a part, which parts won't get updated by themselves, etc. setup.sh hides all that from the operator and makes bad assu...Setting up a real bwauth shouldn't hide steps from the operator. It's important the operator knows what software plays a part, which parts won't get updated by themselves, etc. setup.sh hides all that from the operator and makes bad assumptions on top (it forces installation of deprecated dependencies that touch the network like an old version of pip and peep which has also been deprecated gets installed needlessly as well).Tom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24049remove unnecessary imports and dead code from Torflow's BwAuthority2020-06-13T16:19:48ZTracremove unnecessary imports and dead code from Torflow's BwAuthorityCurrently, BwAuthority code imports stuff which are never
used as well has having code which isn't used.
**Trac**:
**Username**: ewongCurrently, BwAuthority code imports stuff which are never
used as well has having code which isn't used.
**Trac**:
**Username**: ewongTom Rittertom@ritter.vgTom Rittertom@ritter.vghttps://gitlab.torproject.org/legacy/trac/-/issues/24010Make bandwidth authorities use DNS, not IP addresses2020-06-13T16:19:47ZteorMake bandwidth authorities use DNS, not IP addressesCurrently, bandwidth authorities don't see DNS timeouts, because they use IP addresses to connect to bandwidth servers.
We should encourage bandwidth authority operators to use DNS instead, so we down-rate exits with poor DNS connectivity.Currently, bandwidth authorities don't see DNS timeouts, because they use IP addresses to connect to bandwidth servers.
We should encourage bandwidth authority operators to use DNS instead, so we down-rate exits with poor DNS connectivity.Aaron GibsonAaron Gibsonhttps://gitlab.torproject.org/legacy/trac/-/issues/23412Do bandwidth authorities reset history when the relay fingerprint changes?2020-06-13T16:19:47ZTracDo bandwidth authorities reset history when the relay fingerprint changes?I deleted all the private keys from my relay and created a new one, within the day of the new one running it’s middle probability jumped up to around what the other relay had after a week of running. Is this a bug?
My relay.
https://at...I deleted all the private keys from my relay and created a new one, within the day of the new one running it’s middle probability jumped up to around what the other relay had after a week of running. Is this a bug?
My relay.
https://atlas.torproject.org/#details/CD8F43AE828C7F9A8B32C6DC9B5D7DABB4D27A97
**Trac**:
**Username**: Dbryrtfbcbhgfhttps://gitlab.torproject.org/legacy/trac/-/issues/22748Show which bandwidth authority files are used for which percentiles2020-06-13T16:19:45ZteorShow which bandwidth authority files are used for which percentilesIt would be easier to assess the impact of changes like #22744 if we could see which percentiles are using which bwauth files.It would be easier to assess the impact of changes like #22744 if we could see which percentiles are using which bwauth files.https://gitlab.torproject.org/legacy/trac/-/issues/22744Remove the smaller bandwidth authority files2020-06-13T16:19:45ZteorRemove the smaller bandwidth authority filesSmall bwfiles make it easier for relays to get stuck in a low measurement bucket (#22741). And they also mean that the lower buckets measure latency more than bandwidth.
I suggest we remove at least the 32k and 64k files.Small bwfiles make it easier for relays to get stuck in a low measurement bucket (#22741). And they also mean that the lower buckets measure latency more than bandwidth.
I suggest we remove at least the 32k and 64k files.https://gitlab.torproject.org/legacy/trac/-/issues/22741Make a tool that sends bandwidth to relays stuck with low measurements2020-06-13T16:19:44ZteorMake a tool that sends bandwidth to relays stuck with low measurementsSome relays get stuck in a low bandwidth authority bucket, and never get out. And their self-measurement isn't enough to unstick them (#22453).
So we want a tool that sends traffic through relays with low consensus weight, ideally enoug...Some relays get stuck in a low bandwidth authority bucket, and never get out. And their self-measurement isn't enough to unstick them (#22453).
So we want a tool that sends traffic through relays with low consensus weight, ideally enough to guarantee them the Fast flag (at which point, the client bandwidth allocation and network measurement take over).
It would be nice to just send drop cells.
But I think we can do this using stem and tor and some URL downloading library.
Sticking this in "Torflow" because that's where it might end up.
But maybe it would be better in its own component eventually.https://gitlab.torproject.org/legacy/trac/-/issues/22031Write a bandwidth server file creator and file checker2020-06-13T16:19:43ZteorWrite a bandwidth server file creator and file checkerWe have instructions for creating bandwidth server files, but we could easily do it using a script.
It would also be great to have another python script that connects to the bandwidth server, and makes sure all the files are present and...We have instructions for creating bandwidth server files, but we could easily do it using a script.
It would also be great to have another python script that connects to the bandwidth server, and makes sure all the files are present and the right size.https://gitlab.torproject.org/legacy/trac/-/issues/21995Set PreferIPv6 on Bandwidth Scanner SOCKSPorts2020-06-13T16:19:43ZteorSet PreferIPv6 on Bandwidth Scanner SOCKSPortsTor Browser sets this option, so the bandwidth authorities should set it too. This only matters when using a bandwidth server with a DNS with IPv6 addresses.Tor Browser sets this option, so the bandwidth authorities should set it too. This only matters when using a bandwidth server with a DNS with IPv6 addresses.https://gitlab.torproject.org/legacy/trac/-/issues/21991Allow bandwidth scanners to use non-Exits to access onion addresses2020-06-13T16:19:42ZteorAllow bandwidth scanners to use non-Exits to access onion addressesTor's bandwidth scanner passes the bandwidth server HTTPS address to Tor's SOCKSPort. So using an onion address should just work.
But to get the full benefit from onion addresses, we need to do two things:
* run the bandwidth server as ...Tor's bandwidth scanner passes the bandwidth server HTTPS address to Tor's SOCKSPort. So using an onion address should just work.
But to get the full benefit from onion addresses, we need to do two things:
* run the bandwidth server as a single onion service (otherwise the variance is too high), and
* allow the second relay in the path to be a middle or guard when connecting to onion addresses.https://gitlab.torproject.org/legacy/trac/-/issues/21990Use a sensible default set of bandwidth servers2020-06-13T16:19:42ZteorUse a sensible default set of bandwidth serversThe default bandwidth authority HTTPS server is easy to overload by mistake, and has no redundancy.
Ideally, we should specify a default list with:
* An IPv4 address,
* An IPv6 address,
* A DNS address (ideally on a CDN with dual-stack ...The default bandwidth authority HTTPS server is easy to overload by mistake, and has no redundancy.
Ideally, we should specify a default list with:
* An IPv4 address,
* An IPv6 address,
* A DNS address (ideally on a CDN with dual-stack support)
* An onion address (once we test this works)https://gitlab.torproject.org/legacy/trac/-/issues/21846BwAuthority can't be run out of the box without manual work2020-06-13T16:19:42ZTracBwAuthority can't be run out of the box without manual workCurrently the bwauthority is quite out of date with the current python ecosystem and doesn't play nice in a dev environment.
The submitted patch makes the following updates:
1) Removes the cron setup to a separate script (so running s...Currently the bwauthority is quite out of date with the current python ecosystem and doesn't play nice in a dev environment.
The submitted patch makes the following updates:
1) Removes the cron setup to a separate script (so running setup.sh doesn't alter your crontab)
2) Changes the shell script to use a modern pip instead of peep (peep is deprecated and hash checking is incorporated into pip as of version 8.0 (https://pypi.python.org/pypi/peep/)
3) Updated dependencies as far as possible (Elixer's last version is 0.7.1 and is only compatible with SQLAlchemy <= 0.7.8, obvious future enhancement to move away from Elixir)
4) Added a convenience script to kill all the spun up processes
5) Updated the ReadMe and the .gitignore to reflect these changes.
**Trac**:
**Username**: davidwfhttps://gitlab.torproject.org/legacy/trac/-/issues/21698Add a 128M file for really fast relays2020-06-13T16:19:40ZteorAdd a 128M file for really fast relaysAaron GibsonAaron Gibson