Think about using `self._relays` in `_relays_with_flag()` and friends

When we currently call _relays_with_flag() and friends the self.relays property might cause a refresh of relays. This works fine in general but is a) expensive and b) it causes hangs in unit tests in case one uses the property without freeze_time() or even just instantiates a RelayList.

We should think whether we can use self._relays instead. If not in every use case maybe it's possible to use it in some of those cases, though.