Find a better solution to setting measurement timeout in long running tests
In the bridge_reachability test by default we wait for 120 seconds for tor to reach 100% bootstrapping, before considering the test to have failed. The problem with this is that by default the measurement timeout is set to only 60 seconds.
This means that if the default measurement timeout is left untouched the measurement will be killed (and considered to have failed), before the test has a time to reach it's own 120 second timeout.
This is currently solved by checking in the
setUp that the advanced->measurement_timeout is >= whatever timeout is set inside of the bridge_reachability test. If this is not the case then an error message is printed:
[!] The measurement timeout is less than the bridge reachability test timeout [!] Adjust your ooniprobe.conf file by setting the advanced: measurement_timeout: value to 120
The problem with this is that in a default install the test will not work unless the user edits their config file accordingly.
There are two ways that this can be improved:
Allowing tests to override the global advanced->measurement_timeout value. This would be quite simple to implement, but it would have as a side effect the fact that if the user is running more than one test at the same time, because they are all part of a deck for example, also their timeout would be increased. This is especially problematic, because this behaviour is something that the user doesn't really control. I would therefore advise against this option.
Make some changes to the measurement scheduling engine and making the test timeout parameter be something that is not global, but that can also be set on a test by test basis. I think this is the ideal option, but it requires a little bit more of work.