Manage Ganeti cluster domain secret in Puppet
The cluster domain secret, stored in /var/lib/ganeti/cluster-domain-secret on all nodes of a Ganeti cluster, must be identical between all nodes involved in a cross-cluster migrations. Furthermore, the upstream documentation offers this recommendation:
It is recommended to assign the same domain secret to all clusters of the same security domain, so that instances can be easily moved between them.
The documentation suggests using gnt-cluster renew-crypto --cluster-domain-secret=/.../ganeti.cds to install the new secret, but it seems like simply dropping the file content into place is sufficient. gnt-cluster verify will only complain if the file not identical between nodes.
Activity
added Backlog label
assigned to @lavamind
mentioned in commit wiki-replica@97c3a82b
could you expand on why we need to manage this in puppet specifically?
The documentation suggests using
gnt-cluster renew-crypto --cluster-domain-secret=/.../ganeti.cdsto install the new secret, but it seems like simply dropping the file content into place is sufficient.gnt-cluster verifywill only complain if the file not identical between nodes.well, in the documentation you deleted in wiki-replica@97c3a82b, it seems that we need to first stop ganeti, change the secret, and start it again... so it's not quite just a drop-in...
could you expand on why we need to manage this in puppet specifically?
So we don't forget if/when we create a new cluster.
well, in the documentation you deleted in wiki-replica@97c3a82b, it seems that we need to first stop ganeti, change the secret, and start it again... so it's not quite just a drop-in...
It might be safer to do that, but I don't think it's required. The upstream documention doesn't mention it. I did stop the ganeti daemon on the master node, but not on the other nodes, and yet the transfer went ahead just fine.
-
Jérôme Charaoui commented on a discussion: #41159 (comment 2900743)
could you expand on why we need to manage this in puppet specifically?
So we don't forget if/when we create a new cluster.
would that mean all clusters would share an identical secret?
well, in the documentation you deleted in wiki-replica@97c3a82b, it seems that we need to first stop ganeti, change the secret, and start it again... so it's not quite just a drop-in...
It might be safer to do that, but I don't think it's required. The upstream documention doesn't mention it. I did stop the ganeti daemon on the master node, but not on the other nodes, and yet the transfer went ahead just fine.
ah, but you did stop it... so maybe it's required on the master?
i would still keep a note in the docs about this to avoid future head scratching...
...
On 2023-05-08 15:11:36, Jérôme Charaoui (@lavamind) wrote:
-
would that mean all clusters would share an identical secret?
Yes.
ah, but you did stop it... so maybe it's required on the master?
Maybe. I'll do a test when implementing this, by cycling the secret and attempting a migration.
i would still keep a note in the docs about this to avoid future head scratching...
Done.
-
This makes me queasy. It's okay as a one-off thing when migrating to a new cluster and shutting down the old one, but in the case we're interested in here (migrating between gnt-fsn to gnt-dal), I think i'd rather have the secrets differ for security reasons.
I must admit I'm not clear exactly on why this could be a problem, but it seems problematic to have the same secrets everywhere, as a ground rule.
In fact, I think we should change the procedure to explicitly have a step to reset the credentials on the old cluster after the migration is completed...
...
On 2023-05-08 15:56:08, Jérôme Charaoui (@lavamind) wrote:
Jérôme Charaoui commented on a discussion: #41159 (comment 2900766)
would that mean all clusters would share an identical secret?
Yes.
-
Since the secret is only used to verify data that's in transit between clusters, and not in other scenarios, I think its fine. That's what upstream recommends; I trust them to have thought this through.
If we want the extra security, I'd suggest we cycle the cluster domain secret right before initiating any cross-cluster migrations, that way it would make a replay attack much harder in case the secret ever leaked somehow, during normal operations.
mentioned in commit wiki-replica@fa608330
-
For the record, the command
gnt-cluster renew-crypto --cluster-domain-secret=/.../ganeti.cdsdoes not actually work at all, this was one of the issues reported by @anarcat in https://github.com/ganeti/ganeti/issues/1696 mentioned in commit wiki-replica@937fb8fd