Make the test suite clean up after itself even in most tricky failure modes
Originally created by @intrigeri on #17216 (Redmine)
Goals:
- avoid leaving zombie processes behind (e.g. Xvfb, unclutter, tor processes started by Chutney, avconv, local services run by cucumber, etc.)
- drop the whole “reboot before running the test suite” dance in our CI (tails/sysadmin#10601, tails/sysadmin#11295)
- have CI nodes that can run both builds & tests, which would provide great performance improvements to our feedback loop: we often have build jobs in the queue while isotesters are idling, or test jobs in the queue while builders are idling
How to do so? Wrap the test suite with something that will track resources and clean them up, such as:
-
systemd-run
, which could a not-super-expensive incremental step forward: https://gitlab.tails.boum.org/tails/sysadmin/-/issues/11295#note_61201 - Docker, which would allow us to have multiple Jenkins executors on one single big node (instead of a number of isotesterN VMs with one executor each), resulting in better usage of our hardware resources:
Scope:
- Initially: our CI.
- Ideally, the solution would also work for developers who run the test suite locally.
- Ideally, the solution would not depend on Jenkins much, if at all, so we benefit from it even if we migrate to another kind of CI some day.
Note: For S11, this fits in:
-
D.1 - Make Tails development process more efficient
: The possibility of starting test runs in an unclean state allows for the arising of new problems that may be hard to debug, thus potentially making the work of developers more difficult.
Status
Let’s not bother until we know whether we migrate to GitLab CI.
Edited by intrigeri