conflux: Support joining rendezvous circuits at the service
Note: at the time of writing arti main is at 08c00299
There are at least two issues preventing us from implementing onion service conflux.
Stream handling
The tunnel reactor (be it multi-path or single-path) has a CellHandlers
structure that contains a handler for incoming stream requests. When an incoming
stream request handler is added to a multi-path tunnel's reactor, the handler is
shared by all the circuits in the tunnel. The caller of
ClientCirc::allow_stream_requests() gets a Stream of IncomingStreams, each
of which comes from one of the circuit legs in the tunnel (so different
IncomingStreams on the same tunnel may come from different circuit legs). The
leg the IncomingStream originated from is specified in the StreamReqInfo,
and is used as the leg of the HopLocation of the resulting StreamTarget.
This HopLocation is then used in the various CtrlMsgs sent by the
StreamTarget to the reactor (such as ClosePendingStream).
For each accepted incoming stream, a stream gets added to the stream map of the
circuit leg the stream request came on. The stream maps are then polled via
ConfluxSet::next_circ_action(), and any resulting cells that need to be sent on
the tunnel that count towards the conflux seqnos will get rerouted to the
current primary leg.
When a cell is received on one of the circuits, on the other hand, things start to break... Every new cell (message) will get delivered to a stream on the same leg it was received on. This is not quite right though, because the other end of the circuit (the onion service client) might choose to send cells on a different leg than the leg the stream was initially opened on... For example, the following interleaving wouldn't work with the current impl (for multiple reasons!):
- client creates two rend circuits (at two separate rend points) to an onion service
- client and service complete the conflux handshake
- clients sends BEGIN on circuit 1, followed by some DATA cells
- client sends a SWITCH, and then continues sending stream data for the same stream, but this time on circuit 2 (its new primary circuit)
- the service closes the circuit, because it will try
(and fail) to find the open stream entry in one of the
CircHops of circuit 2 ("Cell received on nonexistent stream!?")
In fact, you'll notice this tunnel reactor bug affects all conflux circuits,
not just the rendezvous ones. It happens because ConfluxSet::next_circ_action()
always routes any input message for handling on the Circuit leg it originated
from, which may not have the necessary state to handle the cell if the sender
SWITCH-ed in between the time the stream was opened and the receipt of the cell:
select_biased! {
// Check whether we've got an input message pending.
ret = input.next().fuse() => {
let Some(cell) = ret else {
return Ok(CircuitAction::RemoveLeg {
leg: leg_id,
reason: RemoveLegReason::ChannelClosed,
});
};
Ok(CircuitAction::HandleCell { leg: leg_id, cell })
},
<snip>
}Joining rendezvous circuits
On the service side, we currently have no way of joining circuits. When handling
a LINK cell in the reactor, there is no way to communicate to the service that
the circuit the cell was received on may be a rendezvous circuit that needs to
be linked with another rendezvous circuit. This is a problem, because our
implementation relies on the user of the circuit reactor (currently
ClientCirc) handling its merging with another reactor to form a multipath
tunnel (via the ShutdownAndReturnCircuit and LinkCircuits control messages).
I believe we're going to have a similar problem for exits.