Instead of telling the user "when clean_conflicts asks you this, tell
it to remove those files", just remove the damn files.

That removes 13 prompts!

That "howto" section was getting really unwieldy.

Before:

 - Modifying an existing configuration
 - Adding a new module
 - Contributing changes back upstream
 - Running tests
 - Validating Puppet code
 - Listing all hosts under puppet
     - Other ways of extracting a host list
 - Running Puppet everywhere
 - Batch jobs on all hosts
 - Progressive deployment
 - Adding/removing a global admin
 - Examining a Puppet catalog
     - List resources by type
     - View/filter full catalog
 - Troubleshooting
 - Consult the logs of past local Puppet agent runs
     - Running Puppet by hand and logging
     - Finding exported resources with SQL queries
     - Finding exported resources with PuppetDB
 - Password management
 - Getting information from other nodes
     - Exported resources
     - PuppetDB lookups
     - Puppet query language
     - LDAP lookups
     - Hiera lookups
 - Revoking and generating a new certificate for a host
 - Pager playbook
     - [catalog run: PuppetDB warning: did not update since \...\]
     - Problems pushing to the Puppet server
 - Disaster recovery

31 entries, 17 top level, 14 second.

After:

 - Programming workflow
     - Modifying an existing configuration
     - Adding a new module
     - Contributing changes back upstream
     - Running tests
     - Validating Puppet code
 - Puppet tricks
     - Password management
     - Getting information from other nodes
         - Exported resources
         - PuppetDB lookups
         - Puppet query language
         - LDAP lookups
         - Hiera lookups
     - Revoking and generating a new certificate for a host
 - Deployments
     - Listing all hosts under puppet
     - Other ways of extracting a host list
     - Running Puppet everywhere
     - Batch jobs on all hosts
     - Progressive deployment
     - Adding/removing a global admin
 - Troubleshooting
     - Consult the logs of past local Puppet agent runs
     - Running Puppet by hand and logging
     - Finding exported resources with SQL queries
     - Finding exported resources with PuppetDB
     - Examining a Puppet catalog
         - List resources by type
         - View/filter full catalog
 - Pager playbook
     - [catalog run: PuppetDB warning: did not update since \...\]
     - Problems pushing to the Puppet server
 - Disaster recovery

34 entries, 6 top level, 23 second level, 7 third level

The caveat here is that the "third level" here is actually a *fourth*
level, something which we're typically trying to avoid, but in this
case this is not really a possibility.

and puppet itself takes care of reloading multipath when adding a new
alias

Instead of mucking around with the awful iscsi target names we're
stuck with, just go ahead and rescan all the LUNs. There doesn't seem to
be a difference in the delay between rescanning a signal target versus
all targets (almost none).

Also, the previous version failed to include the instructions to
rescan/resize on *all* the cluster nodes, because that doesn't happen
automatically and if the instance is happens to be restarted on another
node, it could fail to boot or worse, cause io errors.

The rationale here is similar: we list the linux kernel packages and
typically show a bunch of "residual config" packages which we then
manually clean up. That's silly, we should just remove then.

Autoremove is also similar: we're going to cleanup a bunch of stuff
after the upgrade, might as well *not* upgrade those packages in the
first place. This should make the upgrade slightly faster.

We tell people to look for those files but not what to do with
them. That is what we do with them. We clean them up.

That said, clean_conflicts doesn't handle exactly the same file set,
but that's likely to be fixed in the (eventual?) rewrite.

A few of those steps might actually clean some of the stuff the nagios
check will catch.

Prevent non-root users from logging in while the upgrade is in progress.
See pam_nologin.so

This will reduce the noise in subsequent runs.

Those are not present in bullseye and somehow do not get cleaned up on
dist-upgrade.

That df output can zip by really fast on a fast uplink, and is
therefore lost in the noise. While we can scrollback to see it, we
need to know about it.

We also just remove that confusing "-o APT::Get::Trivial-Only=true"
blob. It's unclear what it does from the outside. It was originally
taken from the official upgrade procedure as a mechanism to show how
much disk would be used by the upgrade, but the download steps we use
already show that information anyways, so it's actually a duplicate.

We don't need a pause here, what follows below is not the full
upgrade, only the download. And if *that* fails, it's pretty harmless
as it just requires an `apt clean`.

We don't need more interactions in this already long process.

This reverts commit 55ceff5e.

This removes the annoying quotes around all output lines.

This was previously used to track which machine had been upgraded, but
it's really error prone. Just doing this, I found at least one machine
where we forgot to update this file (gayi, still marked as stretch).

That information is also available in PuppetDB (and more accurate)
anyways. It's the datasource we use now to do reports on the upgrade
progress.

To track which host is running a given release, run this on the
PuppetDB host (currently pauli):

    curl -s -G http://localhost:8080/pdb/query/v4 --data-urlencode 'query=nodes { facts { name = "lsbdistcodename" and value = "buster" }}' | jq -r .[].certname

The above will show "buster" nodes, for example.

It's also pretty useless: we could, in theory, use this to (say)
acknowledge all problems matching a given OS, but in practice I have
never done this in three years so far, and it's not because I didn't
know about this group.

So, automation wins here: less churn and manual changes is good. If we
eventually reimplement this in Puppet, we could, in theory, restore
this group, but I don't even think *that* is worth it.

See also team#32901.