chi-node-14 out of disk space on /srv partition
@ahf reported in irc
<ahf> there is an issue with CI on chi-node-14-main: https://gitlab.torproject.org/mikeperry/tor/-/jobs/300052
<ahf> it looks like it's out of disk
and sure enough, the /srv partition is out of disk space
Filesystem Size Used Avail Use% Mounted on
udev 756G 0 756G 0% /dev
tmpfs 152G 2.1M 152G 1% /run
/dev/mapper/vg_ganeti-root 30G 17G 12G 58% /
tmpfs 756G 696K 756G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 512M 0 512M 0% /tmp
/dev/md0 485M 113M 343M 25% /boot
/dev/mapper/vg_ganeti-srv 197G 195G 0 100% /srv
/dev/mapper/vg_ganeti-shadow 492G 115G 377G 24% /srv/shadow
tmpfs 152G 0 152G 0% /run/user/0
i ran ncdu
to see if there's something obvious we could trash for more disk space, but it's just docker volumes. we might be able to buy some time by trashing the volumes (only if they aren't important), but we'll probably need to grow the disk soon anyway.