Failed disk on fsn-node-01

The ganeti node fsn-node-01 has suffered a disk failure this morning:

Jun 21 11:41:46 fsn-node-01 kernel: nvme nvme0: controller is down; will reset: CSTS=0xffffffff, PCI_STATUS=0xffff
Jun 21 11:41:46 fsn-node-01 kernel: nvme 0000:01:00.0: can't change power state from D3cold to D0 (config space inaccessible)
Jun 21 11:41:46 fsn-node-01 kernel: nvme nvme0: Removing after probe failure status: -19
Jun 21 11:41:46 fsn-node-01 kernel: blk_update_request: I/O error, dev nvme0n1, sector 2097168 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Jun 21 11:41:46 fsn-node-01 kernel: md: super_written gets error=-5
Jun 21 11:41:46 fsn-node-01 kernel: md/raid1:md126: Disk failure on nvme0n1p3, disabling device.
                                    md/raid1:md126: Operation continuing on 1 devices.
Jun 21 11:41:46 fsn-node-01 kernel: blk_update_request: I/O error, dev nvme0n1, sector 2097168 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Jun 21 11:41:46 fsn-node-01 kernel: md: super_written gets error=-5
Jun 21 11:41:51 fsn-node-01 kernel: md/raid1:md127: Disk failure on nvme0n1p2, disabling device.
                                    md/raid1:md127: Operation continuing on 1 devices.

The raid1 volumes remain functional, but degraded.