Issues with grub install after update
have been some issues with machines not being able to come back after a reboot because of grub being updated with unattended upgrades and not being properly installed.
@weasel was able to get Alberti back by installing grub on the mounted disk.
root@fsn-node-01:~# gnt-instance activate-disks alberti.torproject.org fsn-node-07.torproject.org:disk/0:/dev/drbd6 fsn-node-07.torproject.org:disk/1:/dev/drbd7 root@fsn-node-07:/# kpartx -a -v -p '-p' -v /dev/drbd6 add map drbd6-p1 (253:71): 0 41940992 linear 147:6 2048 root@fsn-node-07:/# mkdir /TARGET root@fsn-node-07:/# mount /dev/mapper/drbd6-p1 /TARGET root@fsn-node-07:/TARGET# cp -av `readlink -f /dev/drbd6` dev/new-root '/dev/drbd6' -> 'dev/new-root' root@fsn-node-07:/TARGET# cp -av `readlink -f /dev/mapper/drbd6-p1` dev/new-root1 '/dev/dm-71' -> 'dev/new-root1' root@fsn-node-07:/TARGET# chroot . grub-install --modules=part_msdos /dev/new-root Installing for i386-pc platform. /proc/devices: fopen failed: No such file or directory /proc/devices: fopen failed: No such file or directory /proc/devices: fopen failed: No such file or directory /proc/devices: fopen failed: No such file or directory /proc/devices: fopen failed: No such file or directory Unknown device "/dev/new-root1": No such device Installation finished. No error reported. cd / umount TARGET kpartx -d -v -p '-p' -v /dev/drbd6
Grub status after upgrade:
root@alberti:~# DEBIAN_FRONTEND=readline apt install --reinstall grub-pc Reading package lists... Done Building dependency tree Reading state information... Done 0 upgraded, 0 newly installed, 1 reinstalled, 0 to remove and 0 not upgraded. Need to get 131 kB of archives. After this operation, 0 B of additional disk space will be used. Get:1 https://mirror.hetzner.de/debian/packages buster/main amd64 grub-pc amd64 2.02+dfsg1-20+deb10u2 [131 kB] Fetched 131 kB in 0s (790 kB/s) [master ebf3bc14] saving uncommitted changes in /etc prior to apt run Author: Peter Palfrader <email@example.com> 1 file changed, 0 insertions(+), 0 deletions(-) Preconfiguring packages ... (Reading database ... 57271 files and directories currently installed.) Preparing to unpack .../grub-pc_2.02+dfsg1-20+deb10u2_amd64.deb ... Unpacking grub-pc (2.02+dfsg1-20+deb10u2) over (2.02+dfsg1-20+deb10u2) ... Setting up grub-pc (2.02+dfsg1-20+deb10u2) ... Configuring grub-pc ------------------- The GRUB boot loader was previously installed to a disk that is no longer present, or whose unique identifier has changed for some reason. It is important to make sure that the installed GRUB core image stays in sync with GRUB modules and grub.cfg. Please check again to make sure that GRUB is written to the appropriate boot devices. If you're unsure which drive is designated as boot drive by your BIOS, it is often a good idea to install GRUB to all of them. Note: it is possible to install GRUB to partition boot records as well, and some appropriate partitions are offered here. However, this forces GRUB to use the blocklist mechanism, which makes it less reliable, and therefore is not recommended. 1. /dev/sda (21474 MB; QEMU_HARDDISK) 2. - /dev/sda1 (21473 MB; /) 3. /dev/sdb (2147 MB; QEMU_HARDDISK) (Enter the items you want to select, separated by spaces.) GRUB install devices: 1 Installing for i386-pc platform. Installation finished. No error reported. Generating grub configuration file ... Found linux image: /boot/vmlinuz-4.19.0-10-amd64 Found initrd image: /boot/initrd.img-4.19.0-10-amd64 Found linux image: /boot/vmlinuz-4.19.0-9-amd64 Found initrd image: /boot/initrd.img-4.19.0-9-amd64 done Processing triggers for man-db (2.8.5-2) ... Scanning processes... Scanning linux images... Running kernel seems to be up-to-date. No services need to be restarted. No containers need to be restarted. No user sessions are running outdated binaries. root@alberti:~#
We should take care of grub before rebooting the remaining machines.
- reboot all machines with grub reinstalled (in progress)
- keep grub away from automated upgrades