Skip to content

Multi architecture runners for applications team

In order to effectively run tests on tor- and mullvad- browser builds, the applications team needs to have available multi-arch and OS runners for our CI.

I have prototyped how that can be done with custom Gitlab runners and have arrived at a list of steps required to set it up.

Here is an overview of the steps:

  1. In the host machines we will need to install libvirt, QEMU and SSH;
  2. For each OS and architecture pair, we will create a base VM. Each VM must have git and ssh server installed and enabled;
  3. The host machine will generate an SSH key pair and the public key will be authorized in the VM;
  4. The host machine will register the base VM using virsh define.
  5. A new runner will be registered in Gitlab with tags for OS and architecture. It will be a custom runner, as defined in the example config.toml.
  6. It is ready to use.

The most involved step of all is setting up the base VM. How it works under the hood is that for each job the base VM is cloned and destroyed at the cleanup step -- using libvirt allows us to use virsh and virt-clone to make this simple (see prepare.sh and cleanup.sh). To execute the jobs, Gitlab enters the VM using SSH with pubkey authentication (see run.sh).

All the code from the server side of the prototype I created are here: https://gitlab.torproject.org/-/snippets/207. For this prototype I created a Windows a MacOS and a Linux VM to learn about the specificities of each OS, you can see some pipelines that used these test runners here: https://gitlab.torproject.org/brizental/tor-browser/-/pipelines/214727.

I would like to get your feedback on my approach to this and on what needs to be worked on in the prototype for us to be able to deploy this in the applications team dedicated runners.

As for how we can collaborate effectively on this, I suggest from the apps team side we take it from step 2 through 5 and then hand it off to TPA. For that to be possible, our users in the build machines need to be added to the libvirt group.

cc @morgan @micah @pierov (also @ahf, since you said you were interested in this work)