Fix another crash-on-no-threadpool bug.

This one happens if for some reason you start with DirPort enabled
but server mode turned off entirely.

Fixes a case of bug 23693; bugfix on 0.3.1.1-alpha.