Changes

Page history
deprecate ./reboot command authored by anarcat's avatar anarcat 
We move towards the fleet task group, and more broadly away from those
custom scripts. In particular, for reboots, we'd like to have a plain
fleet.reboot command that would just do the right thing with all
hosts.
Show whitespace changes
Inline Side-by-side
howto/upgrades.md
View page @ 9ae1bfd0
...@@ -381,17 +381,17 @@ See below for how to handle specific situations. ...@@ -381,17 +381,17 @@ See below for how to handle specific situations.
## Rebooting a single host ## Rebooting a single host
If this is only a virtual machine, and the only one affected, it can If this is only a virtual machine, and the only one affected, it can
be rebooted directly. This can be done with the `fabric-tasks` script be rebooted directly. This can be done with the `fabric-tasks` task
called `reboot`: `fleet.reboot-host`:
./reboot -H test-01.torproject.org,test-02.torproject.org fab -H test-01.torproject.org,test-02.torproject.org reboot-host
By default, the script will wait 2 minutes before hosts: that should By default, the script will wait 2 minutes before hosts: that should
be changed to *30 minutes* if the hosts are part of a mirror network be changed to *30 minutes* if the hosts are part of a mirror network
to give the monitoring systems (`mini-nag`) time to rotate the hosts to give the monitoring systems (`mini-nag`) time to rotate the hosts
in and out of DNS: in and out of DNS:
./reboot -H mirror-01.torproject.org,mirror-02.torproject.org --delay-hosts 1800 fab -H mirror-01.torproject.org,mirror-02.torproject.org reboot-host --delay-hosts 1800
If the host has an encrypted filesystem and is hooked up with Mandos, it If the host has an encrypted filesystem and is hooked up with Mandos, it
will return automatically. Otherwise it might need a password to be will return automatically. Otherwise it might need a password to be
...@@ -435,24 +435,24 @@ This routine should be able to reboot all hosts with a `rebootPolicy` ...@@ -435,24 +435,24 @@ This routine should be able to reboot all hosts with a `rebootPolicy`
defined to `justdoit` or `rotation`: defined to `justdoit` or `rotation`:
echo "rebooting 'justdoit' hosts with a 10-minute delay, every 2 minutes...." echo "rebooting 'justdoit' hosts with a 10-minute delay, every 2 minutes...."
./reboot -H $(ssh db.torproject.org 'ldapsearch -H ldap://db.torproject.org -x -ZZ -b ou=hosts,dc=torproject,dc=org -LLL "(rebootPolicy=justdoit)" hostname | awk "\$1 == \"hostname:\" {print \$2}" | sort -R') --delay-shutdown=10 --delay-hosts=120 fab -H $(ssh db.torproject.org 'ldapsearch -H ldap://db.torproject.org -x -ZZ -b ou=hosts,dc=torproject,dc=org -LLL "(rebootPolicy=justdoit)" hostname | awk "\$1 == \"hostname:\" {print \$2}" | sort -R') reboot-host --delay-shutdown=10 --delay-hosts=120
echo "rebooting 'rotation' hosts with a 10-minute delay, every 30 minutes...." echo "rebooting 'rotation' hosts with a 10-minute delay, every 30 minutes...."
./reboot -H $(ssh db.torproject.org 'ldapsearch -H ldap://db.torproject.org -x -ZZ -b ou=hosts,dc=torproject,dc=org -LLL "(rebootPolicy=rotation)" hostname | awk "\$1 == \"hostname:\" {print \$2}" | sort -R') --delay-shutdown=10 --delay-hosts=1800 fab -H $(ssh db.torproject.org 'ldapsearch -H ldap://db.torproject.org -x -ZZ -b ou=hosts,dc=torproject,dc=org -LLL "(rebootPolicy=rotation)" hostname | awk "\$1 == \"hostname:\" {print \$2}" | sort -R') reboot-host --delay-shutdown=10 --delay-hosts=1800
Another example, this will reboot all hosts running Debian `bookworm`, Another example, this will reboot all hosts running Debian `bookworm`,
in random order: in random order:
./reboot -H $(ssh puppetdb-01.torproject.org "curl -s -G http://localhost:8080/pdb/query/v4 --data-urlencode 'query=inventory[certname] { facts.os.distro.codename = \"bookworm\" }'" | jq -r '.[].certname' | sort -R) fab -H $(ssh puppetdb-01.torproject.org "curl -s -G http://localhost:8080/pdb/query/v4 --data-urlencode 'query=inventory[certname] { facts.os.distro.codename = \"bookworm\" }'" | jq -r '.[].certname' | sort -R)
And this will reboot all hosts with a pending kernel upgrade (updates And this will reboot all hosts with a pending kernel upgrade (updates
only when puppet agent runs), again in random order: only when puppet agent runs), again in random order:
./reboot -H $(ssh puppetdb-01.torproject.org "curl -s -G http://localhost:8080/pdb/query/v4 --data-urlencode 'query=inventory[certname] { facts.apt_reboot_required = true }'" | jq -r '.[].certname' | sort -R) fab -H $(ssh puppetdb-01.torproject.org "curl -s -G http://localhost:8080/pdb/query/v4 --data-urlencode 'query=inventory[certname] { facts.apt_reboot_required = true }'" | jq -r '.[].certname' | sort -R)
And this is the list of all *physical* hosts with a pending upgrade, alphabetically: And this is the list of all *physical* hosts with a pending upgrade, alphabetically:
./reboot -H $(ssh puppetdb-01.torproject.org "curl -s -G http://localhost:8080/pdb/query/v4 --data-urlencode 'query=inventory[certname] { facts.apt_reboot_required = true and facts.virtual = \"physical\" }'" | jq -r '.[].certname' | sort) fab -H $(ssh puppetdb-01.torproject.org "curl -s -G http://localhost:8080/pdb/query/v4 --data-urlencode 'query=inventory[certname] { facts.apt_reboot_required = true and facts.virtual = \"physical\" }'" | jq -r '.[].certname' | sort)
## Rebooting Ganeti nodes ## Rebooting Ganeti nodes
... ...
......