Bandwidth Authority Renovation

This ticket describes the Bandwidth Authority Refactoring Project.

Goals:

• Replace pyTorCtl with stem[1] or txtorcon
• Address Critical Issues
• Address Scalability Issues
• Implement New Feedback Mechanisms

Critical:

• Bandwith Authorities are leaving 10 percent of the network unmeasured https://trac.torproject.org/projects/tor/ticket/13450
• Understand how accurate the bandwidth authority estimates are https://trac.torproject.org/projects/tor/ticket/7177
• The slowest fraction of the network takes 18 hours to complete https://trac.torproject.org/projects/tor/ticket/9795
• BwAuth takes a long time to scan the network https://trac.torproject.org/projects/tor/ticket/13449
• Make a torflow option that starts every testing circuit at a local Tor relay first https://trac.torproject.org/projects/tor/ticket/9762

Decentralize:

• Decentralized measurement for network load balancing https://trac.torproject.org/projects/tor/ticket/5464

Parallelise:

• Split BwAuth children more equally https://trac.torproject.org/projects/tor/ticket/3440
• Increase the number of parallel scanners in a BandwidthAuthority from 4 to 8 https://trac.torproject.org/projects/tor/ticket/9795

Feedback Mechanisms:

• Implement bwauth cap for latency https://trac.torproject.org/projects/tor/ticket/4708
• Implement bwauth cap for TCP socket exhaustion https://trac.torproject.org/projects/tor/ticket/4709
• Bwauth: Use only "slow" measurements for positive feedback https://trac.torproject.org/projects/tor/ticket/4730
• Bandwidth auths should publish average and weighted onionskin failure rates https://trac.torproject.org/projects/tor/ticket/7281
• Detect overtuned exit relays https://trac.torproject.org/projects/tor/ticket/10791

untracked issues?:

• Scalability - how are the BwAuths scaling?
• How long does this take?
• How much longer will it take with 10000 relays?
• Network stability?
• Circuit failure rates?
• Partition detection?

papers:

• EigenSpeed: Secure Peer-to-peer Bandwidth Evaluation https://www.usenix.org/legacy/event/iptps09/tech/full_papers/snader/snader.pdf http://www.cs.ucla.edu/~nrl/CapProbe/

• Requirements and Software Design for a Better Tor Performance Measurement Tool https://research.torproject.org/techreports/torperf2-2013-10-30.pdf

[1] stem is "is a synchronous (threaded) Python library". There is/was? a plan to integrate it with txtorcon[2].

[2] https://www.torproject.org/getinvolved/volunteer.html.en#txtorcon-stemIntegration