Run clean_consdiffmgr() callback on all directories

Right now, clean_consdiffmgr() is tied to server_mode(). But it should run everywhere that's a directory cache at all.