client: ControlPort set doesn't mean you're a client

Splitting from legacy/trac#19665 (moved) to avoid confusion.

This ticket is solely to make a patch so that when the ControlPort is set (Unix or not), we do not consider it as a client with any_client_port_set().