Matt Traudt authored Jun 22, 2018 and juga committed Jun 24, 2018

This reduces the number of copies of relay descriptors we have to keep
in memory and how frequently we stop to refresh those descriptors.

2018-06-06 04:42:03 ERROR Thread-5 scanner.py:225 - dispatch_worker_thread - Unhandled exception in worker thread
Traceback (most recent call last):
  File "/home/pastly/src/simple-bw-scanner/sbws/core/scanner.py", line 223, in dispatch_worker_thread
    return measure_relay(*a, **kw)
  File "/home/pastly/src/simple-bw-scanner/sbws/core/scanner.py", line 138, in measure_relay
    dest = destinations.next()
  File "/home/pastly/src/simple-bw-scanner/sbws/lib/destination.py", line 222, in next
    self._perform_usability_test()
  File "/home/pastly/src/simple-bw-scanner/sbws/lib/destination.py", line 171, in _perform_usability_test
    circ_id = self._cb.build_circuit([None, exit.fingerprint])
  File "/home/pastly/src/simple-bw-scanner/sbws/lib/circuitbuilder.py", line 175, in build_circuit
    for r in path]
  File "/home/pastly/src/simple-bw-scanner/sbws/lib/circuitbuilder.py", line 175, in <listcomp>
    for r in path]
AttributeError: 'NoneType' object has no attribute 'fingerprint'
2018-06-06 04:42:03 ERROR Thread-10 scanner.py:280 - closure - Unhandled exception caught while measuring torbogen: <class 'AttributeError'> 'NoneType' object has no attribute 'fingerprint'

Not currently properly checking that stem is a new enough version.
It needs to be 'git clone'd and installed manually ...

Added in [0] and small fix in [1]

[0]: https://gitweb.torproject.org/stem.git/commit/stem?id=8a065f42952c9dc9762eaebe4d3fa48210783998
[1]: https://gitweb.torproject.org/stem.git/commit/stem?id=60f034ad8b9c3aa48e7e2ecb0a2e159b6ed5bc71

on our own terms. There are apparently cases when Tor won't stop trying
to build a circuit at CircuitBuildTimeout. One such case is when it is
having trouble connecting to the first hop relay.

It seems to wait on TCP to say whether or not the connection is made,
and TCP doesn't come back until after 2 minutes on my machine. That's
just a theory. It could be Tor internally wants to wait 2 minutes before
giving up, but my money is on TCP.

Anyway. Make the logic in build_circuit_impl 10x harder to understand by
doing some "async" waiting ourselves instead of letting stem handle it.
The logic is inspired by the stem code for extending circuits[0].

[0]: https://stem.torproject.org/api/control.html#stem.control.Controller.extend_circuit

GH: closes #134

GH: ref #134

This also fixes some instances where adding a '%s' or similar was
forgotten

May have some runtime bugs still. Needs a little more testing

Hopefully everything works first try