From c31b36085dd73cff37c099dcfac17c3f9a615d4e Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Antoine=20Beaupr=C3=A9?= <anarcat@debian.org> Date: Mon, 3 Oct 2022 16:18:19 -0400 Subject: [PATCH] start documenting cluster migration procedures --- howto/ganeti.md | 48 ++++++++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 46 insertions(+), 2 deletions(-) diff --git a/howto/ganeti.md b/howto/ganeti.md index 85f0af59..e7e92331 100644 --- a/howto/ganeti.md +++ b/howto/ganeti.md @@ -2056,12 +2056,56 @@ Look into logs on the relevant nodes (particularly `/var/log/ganeti/node-daemon.log`, which shows all commands ran by ganeti) when you have problems. +### Migrating a VM between clusters + +The [export/import](https://docs.ganeti.org/docs/ganeti/3.0/html/admin.html#export-import) mechanism can also be used to export and import +VMs one at a time, if only a subset of the cluster needs to be +evacuated. + + 1. find nodes to host the exported VM on the source cluster and the + target cluster; it needs enough disk space in `/srv/ganeti` to + keep a copy of a snapshot of the VM: + + df -h /srv/ganeti + + 2. have the right kernel modules loaded, which might require a + reboot of the source node: + + modprobe dm_snapshot + + 3. on the master of the source Ganeti cluster, export the VM to the + source node, also use `--noshutdown` if you cannot afford to have + downtime on the VM *and* you are ready to lose data accumulated + after the snapshot: + + gnt-backup export -n fsn-node-01.torproject.org test-01.torproject.org + + 4. copy the VM snapshot from the source node to node in the target + cluster: + + scp /srv/ganeti/... root@chi-node-02.torproject.org:/srv/ganeti + + 5. on the master of the target Ganeti cluster, import the VM: + + gnt-backup import -n chi-node-08.torproject.org --src-node=chi-node-02.torproject.org --src-dir=/srv/ganeti/??? test-01.torproject.org + +### Mass migrating instances to a new cluster + +The [move-instance](https://docs.ganeti.org/docs/ganeti/3.0/html/move-instance.html) command can do this. + +TODO: document mass cluster migrations. + ## Disaster recovery If things get completely out of hand and the cluster becomes too unreliable for service, the only solution is to rebuild another one -elsewhere. Since Ganeti 2.2, there is a [move-instance](http://docs.ganeti.org/ganeti/2.15/html/move-instance.html) command to -move instances between cluster that can be used for that purpose. +elsewhere. Since Ganeti 2.2, there is a [move-instance](https://docs.ganeti.org/docs/ganeti/3.0/html/move-instance.html) command to +move instances between cluster that can be used for that purpose. See +the [mass migration procedure](#mass-migrating-instances-to-a-new-cluster) above. + +The [export/import](https://docs.ganeti.org/docs/ganeti/3.0/html/admin.html#export-import) mechanism can also be used to export and import +VMs one at a time, if only a subset of the cluster needs to be +evacuated. See the [migrating a VM between clusters](#migrating-a-vm-between-clusters) procedure above. If Ganeti is completely destroyed and its APIs don't work anymore, the last resort is to restore all virtual machines from -- GitLab