Implement async token bucket for flow-control
One of the things we need for XON/XOFF flow control is a way to limit the rate that we send data on a stream in response to XON messages. Specifically, the OpenStreamEntStream
implements Stream
, and the circuit reactor expects this stream to return a relay message only when flow-control allows it. So our rate limiting must exist in OpenStreamEntStream
, and must be capable of waking a Waker
(stored in OpenStreamEnt::flow_ctrl_waker
) once the rate limit allows us to send bytes again. This OpenStreamEntStream
also needs to simultaneously support sendme-window flow-control (although only one is used on a stream), so this might end up being a bit ugly.
There are some existing token bucket rate limiter crates, some async and some sync, that we could potentially use. But we have some requirements that they might not satisfy:
- Need the ability to update the rate dynamically as we receive XON messages with new rate limits. I think that we don't need the ability to update the rate while we're awaiting the token bucket, since we don't process incoming cells (which might be an XON) while we're also awaiting streams to send cells.
- Need the ability to set a rate of 0 in the case we get an XOFF.
- Must not be tied to a specific async runtime. I noticed that some crates support both tokio and async-std, but we also have plans to support smol, so this probably wouldn't work. We also want our token bucket to be based around
SleepProvider
anyways.
There are probably other things I haven't considered yet.
There are some existing comments about rate limiting in doc/dev/notes/bw-rate-limit.md
, but some of the designs might be too abstract for what I think we need here. But I'll keep the designs in mind, and we can always extend them later for bw rate limits later if we want to.