Clients fail on the 7th rapid SOCKS request to the same HS
If a tor client gets 7 SOCKS connection requests to a hidden service with an uncached descriptor in rapid succession, it launches a HSDir request for each connection. It doesn't wait for the circuit to any of the HSDirs to be built, or wait for a response from any of the HSDirs.
After 6 connections, it fails on the 7th attempted fetch, because it has tried to (rapidly) fetch the descriptor 6 times, and hasn't got it yet. It then fails each of the outstanding SOCKS requests 1-7.
It can do this all in the same second, before any circuits have a chance to be built.
The client then backs off after the descriptor fetch failure, the circuit build attempts succeed, and then the 8th and subsequent requests succeed.
This behaviour is unlikely to be triggered by HTML-based hidden services. There is typiclly 1 connection with an uncached descriptor to load the initial HTML page, then further connections with a cached descriptor load any page resources.
Observed in 0.2.7.0-dev.
tor logs attached for 3 HSDirs, 1 client, and 1 HS in a chutney network hs-100-clients (branch with these changes in chutney upcoming in #15936 (moved))