tor-proto: Add rate-limiting to `DataWriter`
This MR adds rate limiting to the DataWriter
. For now the rate is set very high (u64::MAX
), and in the future we'll change this dynamically as we receive XON/XOFF signals.
There are three parts:
- The token bucket.
- The
AsyncWriter
wrapper that implements rate-limiting. - The changes to
DataWriter
.
Closes #1992 (closed).
There are some related design notes in #102, !2039, and doc/dev/notes/bw-rate-limit.md
.
I think these mainly focus on the problem of rate limiting channels where the rates are shared among many channels. There are some difficult problems here regarding fairness. For our use case where we need to rate limit individual streams in a single direction, we don't need to worry about a lot of this complexity. I think the use cases are different enough that having a simplified rate limiter for stream flow control makes sense. I looked at some other crates, but didn't find any that I was happy with or that met the features we need (like supporting a rate of 0), and that weren't overly complex. We also want something that will work with a SleepProvider
and is not tokio-only.
In addition to the unit tests, I also tested this on the live network by changing the DataWriter
to use:
let rate_limit = 10_000; // bytes per second
socat TCP-LISTEN:9151,fork SOCKS4A:localhost:<public-iperf3-server>:5201,socksport=9150
$ iperf3 -c localhost -p 9151
Connecting to host localhost, port 9151
[ 5] local ::1 port 51466 connected to ::1 port 9151
[ ID] Interval Transfer Bitrate Retr Cwnd
[ 5] 0.00-1.00 sec 10.4 MBytes 87.0 Mbits/sec 1 1.25 MBytes
[ 5] 1.00-2.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 2.00-3.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 3.00-4.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 4.00-5.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 5.00-6.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 6.00-7.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 7.00-8.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 8.00-9.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
[ 5] 9.00-10.00 sec 0.00 Bytes 0.00 bits/sec 0 1.25 MBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 10.4 MBytes 8.70 Mbits/sec 1 sender
[ 5] 0.00-10.16 sec 106 KBytes 85.5 Kbits/sec receiver
Here we can see that the iperf3 server received at a rate of 85 Kbits/sec (unsure if this is kilo or kibi), which is around 10 KB/sec (the rate we set above).
And for let rate_limit = 100_000
we see:
[ ID] Interval Transfer Bitrate Retr
[ 5] 0.00-10.00 sec 8.62 MBytes 7.23 Mbits/sec 1 sender
[ 5] 0.00-10.24 sec 1.04 MBytes 849 Kbits/sec receiver
/cc @dgoulet