Core tests are broken
Something after commit c12bc4d89ef16d78e2cf2ef22363802ec90022ae broke the core tests.
I believe it has something to do with the changes that have been made to runner.py.
Here is a list of things that I noticed that may or may not be related to this issue, but that are for sure not good practice (this is all related to ooni/runner.py):
There is code duplication in processTest(obj, config) and processTestOptions. Looking around processTest appears to no longer be called.
loadTestsAndOptions has become a huge method. I would consider splitting the things that have to do with the legacy API to another place.
The uber deep nesting plague appears to have started to attack parts of the code. In particular https://gitweb.torproject.org/ooni-probe.git/blob/HEAD:/ooni/runner.py#l200, https://gitweb.torproject.org/ooni-probe.git/blob/HEAD:/ooni/runner.py#l238, https://gitweb.torproject.org/ooni-probe.git/blob/HEAD:/ooni/runner.py#l258
If you start nesting beyond one level, you should ask yourself if there is a better way to write it.
We should also not spend any more time on supporting the old API. Every line of code we write to support it is a line of code we have to maintain, that is bad.