Lower Tor Exit-side DNS TImeout
Tor exits currently wait 5 seconds before deciding to time out a DNS request. It is overwhelmingly likely that this is the cause of the performance issue we saw in tpo/metrics/analysis#33076 (comment 2720122)
The performance degradation there exactly matches the behavior we would see if relays become overloaded and drop UDP packets containing DNS queries, triggering a timeout and retry, at 5 second intervals. When coupled with the fact that shadow doesn't do DNS, and could not reproduce the issue, makes it overwhelmingly likely that DNS timeouts are the culprit here
Even, if by some dark curse, there is another 5 second timeout somewhere else in Tor that contributed to that issue, we definitely know that waiting 5 seconds for DNS on the modern internet is a bit too long.
Lowering the Exit-side DNS timeout to 1 second (or lower) will make this issue impact UX much less. The actual solution is to emit an overload signal when this happens, as per #40222 (closed), and then use sbws to reduce the weights on such relays until the overload signal disappears.
See also #40222 (comment 2727445)