Skip to content

tor-proto: Add rate-limiting to `DataWriter`

This MR adds rate limiting to the DataWriter. For now the rate is set very high (u64::MAX), and in the future we'll change this dynamically as we receive XON/XOFF signals.

There are three parts:

  1. The token bucket.
  2. The AsyncWriter wrapper that implements rate-limiting.
  3. The changes to DataWriter.

Closes #1992 (closed).

There are some related design notes in #102, !2039, and doc/dev/notes/bw-rate-limit.md. I think these mainly focus on the problem of rate limiting channels where the rates are shared among many channels. There are some difficult problems here regarding fairness. For our use case where we need to rate limit individual streams in a single direction, we don't need to worry about a lot of this complexity. I think the use cases are different enough that having a simplified rate limiter for stream flow control makes sense. I looked at some other crates, but didn't find any that I was happy with or that met the features we need (like supporting a rate of 0), and that weren't overly complex. We also want something that will work with a SleepProvider and is not tokio-only.

In addition to the unit tests, I also tested this on the live network by changing the DataWriter to use:

let rate_limit = 10_000; // bytes per second
socat TCP-LISTEN:9151,fork SOCKS4A:localhost:<public-iperf3-server>:5201,socksport=9150
$ iperf3 -c localhost -p 9151
Connecting to host localhost, port 9151
[  5] local ::1 port 51466 connected to ::1 port 9151
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec  10.4 MBytes  87.0 Mbits/sec    1   1.25 MBytes       
[  5]   1.00-2.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   2.00-3.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   3.00-4.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   5.00-6.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   6.00-7.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   7.00-8.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
[  5]   9.00-10.00  sec  0.00 Bytes  0.00 bits/sec    0   1.25 MBytes       
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  10.4 MBytes  8.70 Mbits/sec    1             sender
[  5]   0.00-10.16  sec   106 KBytes  85.5 Kbits/sec                  receiver

Here we can see that the iperf3 server received at a rate of 85 Kbits/sec (unsure if this is kilo or kibi), which is around 10 KB/sec (the rate we set above).

And for let rate_limit = 100_000 we see:

[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  8.62 MBytes  7.23 Mbits/sec    1             sender
[  5]   0.00-10.24  sec  1.04 MBytes   849 Kbits/sec                  receiver

/cc @dgoulet

Edited by opara

Merge request reports

Loading