new-machine-hetzner-robot.mdwn

How to install a new bare metal server at Hetzner
=================================================

Order
-----

 1. get approval for the server, picking the specs from the [main
 website](https://www.hetzner.com/)

 2. head to the [order page](https://robot.your-server.de/order) and pick the right server. pay close
    attention to the location, you might want to put it alongside
    other TPO servers (or not!) depending on redundancy or traffic
    requirements. Click `Add to shopping cart`, leaving all other
    fields as default.

 3. in the `Server login details` page, you should leave `Type` set to
    `Public key`. If you do not recognize your public SSH key in
    there, head to the [server list](https://robot.your-server.de/server) and click on [key
    management](https://robot.your-server.de/key/index) to add your public keys

 4. when you're certain of everything, click `Checkout` in the cart,
    review the order again and click `Order in obligation`.

A confirmation email will be sent by Hetzner at the TPA alias when the
order is filed. Then you wait for the order to complete before being
able to proceed with the install.

Ordering physical servers from Hetzner can be very fast: we've seen 2
minutes turn around times.

Install
-------

At this point you should have received an email from Hetzner with a
subject like:

    Subject: Your ordered SX62 server

It should contain the SSH fingerprint, and IP address of the new host
which we'll use below.

 1. login to the server using the IP address and host key hash
    provided above:

        ssh -o FingerprintHash=md5 -o UserKnownHostsFile=/dev/null root@159.69.63.226

    Note: the `FingerprintHash` parameter above is to make sure we
    match the hashing algorithm used by Hetzner in their email, which
    is, at the time of writing, MD5 (!). Newer versions of SSH will
    also encode the hash as base64 instead of hexadecimal, so you
    might want to decode the base64 into the latter using this: The
    `UserKnownHostsFile` is to make sure we don't store the
    (temporary) SSH host key.

        perl -MMIME::Base64 -e '$h = unpack("H*", decode_base64(<>)); $h =~ s/(..)(?=.)/\1:/g; print $h, "\n"'

 2. Partition disks. This might vary wildly between hosts, but in
    general, we want:

      * GPT partitionning, with space for a 8MB grub partition and
        cleartext `/boot`
      * software RAID (RAID-1 for two drives, RAID-5 for 3, RAID-10
        for 4)
      * crypto (LUKS)
      * LVM, with separate volume groups for different medium (SSD vs
        HDD)

    This can be done with the `tor-install-format-disks` in the
    `tsa-misc` repository, which should be carefully checked and
    configured before running.

 3. Install the system. This can be done with `grml-debootstrap` which
    will also configure grub, a root password and so on. This should
    get you started, assuming the formatted root disk is mounted on
    `/mnt`:

        ROOTPASSWORD=\$(tr -dc 'A-Za-z0-9' < /dev/urandom | head -c 30) \
        grml-debootstrap --grub /dev/sda --target /mnt \
            --hostname $hostname \--release buster \
            --mirror https://mirror.hetzner.de/debian/packages/ \
            --remove-configs --defaultinterfaces

    Note: last time this was ran, `udev` was not installed which led
    to some problems, see [Debian #931235](https://bugs.debian.org/931235).

 4. Once the bootstrapping is complete, you still need to make sure
    the system can boot as, the above doesn't (unfortunately)
    configure everything for you. First, fix the mountpoints:

        editor /mnt/etc/fstab

    For example:

        /dev/mapper/archive01-hdd       /       ext4    defaults,errors=remount-ro      0       1
        UUID=aef7c53c-ed2e-4b9e-b23a-b70a701a2dcb /boot ext4    defaults        0       2
        /dev/mapper/archive01-swap      none    swap    defaults        0       0
        proc           /proc        proc    defaults                      0   0

 5. Review the crypto configuration:

        editor /mnt/etc/crypttab

 6. Do the same with the RAID configuration, probably with something like:

        chroot /mnt sh -c "/usr/share/mdadm/mkconf > /etc/mdadm/mdadm.conf"

 7. Review the network configuration:

        editor /mnt/etc/network/interfaces

    An example safe configuration is:

        iface lo inet loopback

        allow-hotplug eth0
        iface eth0 inet dhcp

 8. Copy paste your key into the root's authorized keys, just to make
    sure you can login:

        cat > /mnt/root/.ssh/authorized_keys

 9. If any of those latter things changed, you need to regenerate the
    initramfs:

        for fs in dev proc run sys  ; do
            mount -o bind /$fs /mnt/$fs
        done
        chroot /mnt update-initramfs -u
        chroot /mnt update-grub
        for fs in dev proc run sys  ; do
            umount /mnt/$fs
        done

 10. Document the LUKS passphrase and root password in `tor-passwords`

 11. Cross fingers and reboot:

        reboot

Configuration
-------------

See [[new-machine]] for post-install configuration steps.