sbws keeps the number of AsyncResults less than the number of threads
This ticket is a follow-up to #28864 (moved).
multiprocessing.Pool can accept any number of queued AsyncResults:
https://docs.python.org/3/library/multiprocessing.html#multiprocessing.pool.Pool
But sbws waits for 5 seconds when there are max_pending_results queued AsyncResults:
https://github.com/torproject/sbws/blob/ee64d76df54ceb3a3c9e1e2a797fd70d68bb0035/sbws/core/scanner.py#L359-L361
This sbws code is unnecessary, because Pool manages its own queue of AsyncResults.
There are two different ways that this code blocks execution:
- when a result finishes, the
time.sleep(5)call blocks the thread from getting a newAsyncResultfor up to 5 seconds - if
max_pending_resultsAsyncResults ever block, the process hangs