Refactor cell scheduling to consider all connections at once
Right now, our cell scheduling works by ensuring that every connection on which any circuit wants to write has at least one cell buffered for writing, and by putting more cells into the connection's write buffer every time a write makes it get low. The circuitmux logic is only used to choose which circuit on a given connection should have permission to write.
But this approach does a bad job in that it treats all connections equally: instead, it should look at all connections which would like to write now, and decide among them.
Reported by Rob Jansen.
Andrea is working on this one. As I understand it, the approach is to decouple the "Put a cell on this connection's output buffer" logic from the two things that trigger it: draining the output buffer a little in response to a libevent event, and making a circuit active on a connection which had no active circuits or buffered cells before. Instead, the main event loop should update a list of connections which would like to have cells put on them, and periodically (every X usec, or every time through the libevent loop, or something like that) queue cells onto the selected connections.
The details above will be a little tricksy. Andrea, could you explain a little more about your approach here? Alternatively I think I can dig up the conversation we had about this stuff, if you're okay with my posting it.