Upgrade infra to Debian 11 (Bullseye)

As per the Roadmap session done during Summit 2020.

Stretch LTS is supported from 6th July 2020 to June 30, 2022.

Note that upgrading directly from Debian N to N+2 is not supported, so one has to first upgrade any Stretch system to Buster, and then (possibly immediately) upgrade it to Bullseye.

Servers

• ant01 (stretch) [intrigeri]
• lizard (stretch)
  • AppArmor blocks "virsh destroy" on lizard: #17903
  • Puppetize the initramfs options in /etc/crypttab.
  • Document that NUMA guest memory needs to be rounded according to huge pages.
• iguana (buster)
• sib (stretch) [intrigeri]
• skink
• stone.tails.boum.org (stretch)

Lizard's VMs

• apt.lizard (stretch)
• apt-proxy.lizard (stretch)
• bitcoin.lizard
• bittorrent.lizard (stretch)
• bridge.lizard (stretch)
• dns.lizard (stretch) -- better to do together with teels so they run the same version of PowerDNS.
• im.lizard (stretch)
• isobuilder*.lizard: tails/sysadmin#17743
• isotester*.lizard (buster)
  • Then notify developers on tails/tails#18837.
• jenkins.lizard (stretch)
  • maybe blocked by tails/sysadmin#17798
  • need to deal with Error: /Stage[main]/Nfs::Client::Debian/Service[nfs-common]/ensure: change from 'stopped' to 'running' failed: Systemd start for nfs-common failed! (seen on jenkins.sib, which does not use NFS, while jenkins.lizard does)
    • hopefully fixed by https://github.com/camptocamp/puppet-nfs/pull/75, which is used by our manifests repo
• mail.lizard (stretch)
  • Re-enable Schleuder lists management (it was temporarily disabled in https://gitlab.tails.boum.org/tails/puppet-tails/-/commit/88be36b97b63922615d282e3546da05029a8cf0f) → sysadmin#17914
• misc.lizard (stretch) -- try to decrease memory after upgrade (1 GB → 512 MB) and check if #17872 happens again.
• puppet.lizard (buster) -- puppetdb is not in Bullseye, and 6.2.0-5 from Sid is buggy.
  • Research and document change needed in /etc/ssl/openssl.cnf to avoid "key size too small" → #17911
  • Pin PuppetDB from sid (it's not available in Bullseye)
  • Puppetize Systemd unit file workaround for startup bug
• puppet-git.lizard (buster)
• rsync.lizard (stretch)
• survey.lizard
• translate.lizard
• whisperback.lizard (stretch)
• www.lizard (stretch) [intrigeri]
  • Pin po4a to buster's version: tails/tails#18667
    • Seems to be done already via tails::website::builder
  • Deal with NFS client Puppet code not supporting Bullseye, same as jenkins.lizard
    • hopefully fixed by https://github.com/camptocamp/puppet-nfs/pull/75, which is used by our manifests repo

Iguana's VMs

• isobuilder*.iguana: tails/sysadmin#17743
• isotester*.iguana (buster)
  • Then notify developers on tails/tails#18837.

sib's VMs

• apt-proxy.sib
• jenkins.sib (stretch): see #17798 [intrigeri]
• isobuilder1.sib (bullseye): tails/sysadmin#17743
• isotester1.sib (buster) [intrigeri]

ant01's VMs

• worker*.ant01 (bullseye): tails/sysadmin#17743

3rd-party VMs

• ecours.tails.boum.org (buster) -- Check versions of Icinga (master, satellites, agents).
• teels.tails.boum.org (stretch) -- Better to do together with dns.lizard so they run the same version of PowerDNS.

Follow-up

• Make "bullseye" the default value for tails::apt::codename
• Remove tails::apt::codename: bullseye in hieradata/node/*.yaml
• Check for obsolete stretch/buster config that can be cleaned up (some stretch aleady done by intrigeri + puppet-tails!87)