Extract information from more kinds of wedged directory connections.
In main.c, we do:
/* This check is temporary; it's to let us know whether we should consider
* parsing partial serverdesc responses. */
if (conn->purpose == DIR_PURPOSE_FETCH_SERVERDESC &&
connection_get_inbuf_len(conn) >= 1024) {
log_info(LD_DIR,"Trying to extract information from wedged server desc "
"download.");
connection_dir_reached_eof(TO_DIR_CONN(conn));We should also, at a minimum, do this for microdesc downloads.
Activity
changed milestone to %Tor: unspecified
See branch
feature_27241, PR at https://github.com/torproject/tor/pull/286(Calling this sponsor8 since it should help save BW)
Trac:
Sponsor: N/A to Sponsor8-can
Status: accepted to needs_review
I'm wondering if we test various kinds of partial descriptor parsing for microdescs and extrainfo.
Particularly handle_response_fetch_microdesc() and router_load_extrainfo_from_string().
Have these functions been exposed to partial downloads and various kinds of fragmented descriptors before?
If not, we should test them for that first. If so, then this is fine by me.
Trac:
Status: needs_review to needs_information-
To help me understand the consequences of this (and to figure out if I need to also do anything here for #25573 (moved)), I have the following questions:
- How do stalled dirconns translate to their linked edge_connection_ts? Can the edge connection "stall" first and/or get closed somewhere else, or will the dirconn always be the thing that stalls?
- What are the consequences to the dir conn if some other code marks the edge connnection is closed first?
- When the dirconn is closed first (such as here or elsewhere in the code), where in the code do we decide to mark/clean up/close the corresponding linked edge conn? I could not find this..
For #25573 (moved), I'm particularly concerned about multihop hsdir dirconns. But if we ever make regular directory activity multihop, this could apply there too. Plus the linked conn close conditions may be relevant to this bug?
-
Replying to mikeperry:
To help me understand the consequences of this (and to figure out if I need to also do anything here for #25573 (moved)), I have the following questions:
- How do stalled dirconns translate to their linked edge_connection_ts? Can the edge connection "stall" first and/or get closed somewhere else, or will the dirconn always be the thing that stalls?
I don't think we have any logic for an edge_connection_t to time out; in princple you can keep an ssh connection running over Tor indefinitely. Directory connections, however, do time out.
- What are the consequences to the dir conn if some other code marks the edge connnection is closed first?
When one of a pair of linked connections is finally closed from connection_unlink in main.c, the other connection gets unlinked from it, and told to read all of its data from its input.
So if something else closes the edge connection, the dir connection will get one last chance to handle any data queued on it before it is closed.
- When the dirconn is closed first (such as here or elsewhere in the code), where in the code do we decide to mark/clean up/close the corresponding linked edge conn? I could not find this..
Hmm. I wonder if nothing closes it, actually. Can you think of some experiment we could do to find out?
For #25573 (moved), I'm particularly concerned about multihop hsdir dirconns. But if we ever make regular directory activity multihop, this could apply there too. Plus the linked conn close conditions may be relevant to this bug?
Replying to mikeperry:
... For #25573 (moved), I'm particularly concerned about multihop hsdir dirconns. But if we ever make regular directory activity multihop, this could apply there too. Plus the linked conn close conditions may be relevant to this bug?
In some cases, bridge clients fetch authority certificates over multihop connections: https://github.com/torproject/tor/blob/94605f08fb89ea79409225362d2fa0f8a07435d7/src/feature/nodelist/routerlist.c#L952
And do most other actions over multihop connections: https://github.com/torproject/tor/blob/48632455a5bd679d5f97c5137f24f91e564abad6/src/feature/dircache/directory.c#L185
And bridge relays upload descriptors over multihop connections: https://github.com/torproject/tor/blob/48632455a5bd679d5f97c5137f24f91e564abad6/src/feature/dircache/directory.c#L379
-
Replying to nickm:
Replying to mikeperry:
To help me understand the consequences of this (and to figure out if I need to also do anything here for #25573 (moved)), I have the following questions:
- When the dirconn is closed first (such as here or elsewhere in the code), where in the code do we decide to mark/clean up/close the corresponding linked edge conn? I could not find this..
Hmm. I wonder if nothing closes it, actually. Can you think of some experiment we could do to find out?
I am not sure. Do we have a way to trigger this bug?
For #25573 (moved), I'm particularly concerned about multihop hsdir dirconns. But if we ever make regular directory activity multihop, this could apply there too. Plus the linked conn close conditions may be relevant to this bug?
I've been digging and I still cannot find anything that closes the edge conn when we mark the dirconn. This is fine for #25573 (moved), since it only deals with edge conns, but if we change that, we'll want to update #25573 (moved).
For purposes of this bug, I think the only question that remains is if we've exposed handle_response_fetch_microdesc() and router_load_extrainfo_from_string() to partial descriptors before.
I think the dirconn closing issue can be an orthogonal bug.
-
Ok I strongly suspect that #8387 (moved) is being caused by us not properly closing edge conns for dirconns underneath. We can use that bug for the dirconn/edgeconn piece, I guess.
-
FTR - this ticket could be merged, but if so I think we should file a new ticket to add some logic to always close underlying edge connections if their dir connections are closed first. We maybe should do that anyway, as that may be one of the things causing #8387 (moved).
moved to tpo/core/tor#27241
