Changes to default ruleset state need to work even if the rule was previously present

This was something we have worked around until now. But when we fix legacy/trac#8774 (moved) we don't want to have to re-name all of the affected rulesets.