---
title: TPA-RFC-8: GitLab CI libvirt exception
---

Summary: create two bare metal servers to deploy Windows and Mac OS
runners for GitLab CI, using libvirt.

# Background

Normally, we try to limit the number of tools we use inside TPA (see
[TPA-RFC-3: tools](policy/tpa-rfc-3-tools)). We are currently phasing out the use of
[libvirt](howto/kvm) in favor of [Ganeti](howto/ganeti), so new virtual machines
deployments should normally use Ganeti on all new services.

GitLab CI (Continuous Integration) is currently at the testing stages
on our [GitLab](howto/gitlab) deployment. We have Docker-based "shared" runners
provided by the F-Droid community which can be used by projects on
GitLab, but those only provide a Linux environment. Those environments
are used by various teams, but for Windows and Mac OS builds,
commercial services are used instead. By the end of 2020, those
services will either require payment (Travis CI) or are extremely slow
(Appveyor) and so won't be usable anymore.

Travis CI, in particular, has deployed a new "points" system that
basically allows teams to run at *most* 4 builds per month, which is
really not practical and therefore breaks MacOS builds for
tor. Appveyor is hard to configure, slow and is a third party we would
like to avoid.

# Proposal

GitLab CI provides a [custom executor](https://docs.gitlab.com/runner/executors/custom.html) which allows operators to
run arbitrary commands to setup the build environment. @ahf figured
out a way to use libvirt to deploy Mac OS and Windows virtual machines
on the fly.

The proposal is therefore to build two (bare metal) machines (in the
Cymru cluster) to manage those runners. The machines would grant the
GitLab runner (and also ahf) access to the `libvirt` environment
(through a role user).

ahf would be responsible for creating the base image and deploying the
first machine, documenting every step of the way in the TPA wiki. The
second machine would be built with Puppet, using those instructions,
so that the first machine can be rebuilt or replaced. Once the second
machine is built, the first machine should be destroyed and rebuilt,
unless we are absolutely confident the machines are identical.

## Scope

The use of libvirt is still discouraged by TPA, in order to avoid the
cognitive load of learning multiple virtualization environments. We
would rather see a Ganeti-based custom executor, but it is considered
to be too time-prohibitive to implement this at the current stage,
considering the Travis CI changes are going live at the end of December.

This should not grant @ahf root access to the servers, but, as per
[TPA-RFC-7: root access](policy/tpa-rfc-7-root), this might be considered, if absolutely
necessary.

# Deadline

Given the current time constraints, this proposal will be adopted
urgently, by Monday December 7th.

# Status

This proposal is currently in the `standard` state.

# References

 * [example of libvirt in the custom executor](https://docs.gitlab.com/runner/executors/custom_examples/libvirt.html)
 * [building a MacOS image for KVM](https://github.com/foxlet/macOS-Simple-KVM)
 * [CI service documentation](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/service/ci)
