Decentralized measurement for network load balancing

EigenSpeed could provide a lot of security improvements to the Tor network in the face of all sorts of amplification attacks. It just sort of sucks because the passive version could not measure fast relays, and so we've never used it.

However, an active version based on CapProbe, PacketPair, etc could possibly measure capacity in as little as a handful of UDP packets, enabling distributed active lightweight measurements. We could also blend in circuit failure rate information.

As an alternative, we could also try using the passive EigenSpeed for slow relays and the bw authorities only for the faster ones...

The big problem is that this is basically a research effort. We're going to need to try at least a couple different versions of these designs and compare them to each other, and then compare them to the current bandwidth authorities, to make sure everything is as performant and abuse tolerant as possible.

added SponsorZ-large in Legacy / Trac component::core tor/torflow in Legacy / Trac owner::arma in Legacy / Trac parent::5992 in Legacy / Trac performance in Legacy / Trac priority::medium in Legacy / Trac severity::normal in Legacy / Trac status::assigned in Legacy / Trac type::enhancement in Legacy / Trac labels

Eigenspeed paper: http://www.usenix.org/event/iptps09/tech/full_papers/snader/snader.pdf http://www.usenix.org/event/iptps09/tech/full_papers/snader/snader_html/

Who knows what black hole the source has disappeared into, though...

Trac:
Cc: arma to arma, aagbsn
Summary: Evaluate Active EigenSpeed/Hybrid Eigenspeed to Evaluate Active EigenSpeed vs Hybrid EigenSpeed vs Existing Bw Auths
Owner: mikeperry to N/A
Status: new to assigned

arma: I am going to assign this to you since you're our Research Rangler. Maybe if you poke people about it, they will actually listen.

Trac:
Owner: N/A to arma

FYI, there are a ton of lightweight capacity measurement mechanisms published since CapProbe that claim to be improvements. Examples include: Packet Twins, TRIO, and "ThroughputIndex" (fully passive).

If you check the CapProbe citation lists, there's also a ton of new work on measuring capacity in overlay networks that directly cites CapProbe: http://ksubrick.ist.psu.edu/showciting?doi=10.1.1.76.8001

The challenge for us is to find not only the best one of these for Tor's current and future transports, but to also blend in latency (legacy/trac#4708 (moved)), circuit failure (legacy/trac#1984 (moved)), tcp socket exhaustion (legacy/trac#4709 (moved)), feedback (legacy/trac#1976 (moved)), and resilience to gaming.

If we have to do both the research and the implementation ourselves, this is a large project. It is also closely related to network performance and security.

Trac:
Keywords: N/A deleted, SponsorZ-large performance added

Trac:
Summary: Evaluate Active EigenSpeed vs Hybrid EigenSpeed vs Existing Bw Auths to Decentralized measurement for network load balancing

Going to re-parent this to legacy/trac#5992 (moved). We may not strictly need this to defend against path bias, anyway.

Trac:
Parent: legacy/trac#5456 (moved) to legacy/trac#5992 (moved)

FYI: I broke off the circuit failure stuff for path bias into legacy/trac#7509 (moved)/legacy/trac#7281 (moved).

Trac:
Cc: arma, aagbsn to arma, aagbsn, isis@torproject.org

Trac:
Cc: arma, aagbsn, isis@torproject.org to arma, aagbsn, isis@torproject.org, robgjansen

Replying to mikeperry:

Eigenspeed paper: http://www.usenix.org/event/iptps09/tech/full_papers/snader/snader.pdf http://www.usenix.org/event/iptps09/tech/full_papers/snader/snader_html/

Who knows what black hole the source has disappeared into, though...

The code for EigenSpeed is here: https://bitbucket.org/hatswitch/eigenspeed. Nikita sent me an old patch when asked about the code used in "Improving Security and Performance in the Tor Network through Tunable Path Selection". Not sure if that one is helpful though...

Trac:
Cc: arma, aagbsn, isis@torproject.org, robgjansen to arma, aagbsn, isis@torproject.org, robgjansen, gk

This ticket is tagged SponsorZ, but it looks like progress stalled two years ago. Is this still a thing that needs funding?

Trac:
Severity: N/A to Normal
Sponsor: N/A to N/A
Reviewer: N/A to N/A

PeerFlow: Secure Load Balancing in Tor http://www.robgjansen.com/publications/peerflow-popets2017.pdf

Trac:
Parent: legacy/trac#5992 (moved) to N/A
Priority: Medium to High

Trac:
Parent: N/A to legacy/trac#5992 (moved)

Trac:
Cc: arma, aagbsn, isis@torproject.org, robgjansen, gk to arma, aagbsn, isis@torproject.org, robgjansen, gk, starlight@binnacle.cx

Priorities and Severities in torflow are meaningless, setting them all to Medium/Normal.

Trac:
Priority: High to Medium

mentioned in issue legacy/trac#5992 (moved)

mentioned in issue legacy/trac#7509 (moved)

mentioned in issue tpo/core/tor#7509 (closed)

moved from legacy/trac#5464 (moved)

added Feature label and removed 1 deleted label

removed 1 deleted label

added Icebox label

Status on this one: since Mike made this ticket, the Peerflow paper became a thing, and Rob even has an implementation of a collaborative flooding approach that he's been using for the recent experiments to exercise each relay.

So in that sense we have made progress on secure bandwidth measurement, but not so much on decentralized bandwidth measurement.

Mike, what did you have in mind here that we haven't achieved?

And cc'ing @gk and @juga so they can follow along, since part of this is about the glorious future of bwauths.

For those not up to speed (no pun intended):

Since the PeerFlow paper, we've taken a step back and designed a system that we think is more reasonable for Tor's threat model and operational environment called FlashFlow (yes, too many "flows" being thrown around).

FlashFlow is basically a system to run my flooding experiment, but distributed across a set of bandwidth authorities with well-defined (and more intelligent) algorithms for coordination, measurement of new relays, etc.

We have a research paper explaining the system and it's security and performance implications, Tor proposal 316 detailing how Tor could adopt it, and @pastly implemented it specifically for Tor deployment.

We had originally intended sbws to be a temporary band-aid and designed FlashFlow to be the future.

removed Feature label

closed

moved to team#195

mentioned in issue team#182 (closed)