@@ -438,15 +438,31 @@ the course of its regular work. Those tend to pile up, unless they get
cleaned. [Upstream suggests](https://docs.gitlab.com/runner/executors/docker.html#clearing-docker-cache) a [fairly naive shell script](https://gitlab.com/gitlab-org/gitlab-runner/blob/master/packaging/root/usr/share/gitlab-runner/clear-docker-cache) to do
this cleanup, but it has a number of issues:
1. it is noisy ([patched locally with this MR](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/2711))
1. it is noisy (tried to [patch this locally with this MR](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/2711), but
was refused upstream)
2. it might be too aggressive
So we only run it weekly, and instead run a more "gentle" `docker
system prune` command to cleanup orphaned stuff after 3 days.
Also note that documentation on this inside GitLab runner is
inconsistent at best, see [this other MR](https://gitlab.com/gitlab-org/gitlab-runner/-/merge_requests/2711) and [this issue](https://gitlab.com/gitlab-org/gitlab-runner-docker-cleanup/-/issues/21).
So we're not using the upstream cleanup script, and we suspect
upstream itself is not using it at all (i.e. on `gitlab.com`) because
it's fundamentally ineffective.
Instead, we have a set of cron jobs (in
`profile::gitlab::runner::docker`) which does the following:
1. clear all volumes and dead containers, daily (equivalent of the
upstream clear-docker-cache for volumes, basically)
2. clear images older than 30 days, daily (unless used by a running
container)
3. clear all dangling (ie. untagged) images, daily
4. clear all "nightly" images, daily
Note that this documentation might be out of date and the Puppet code
should be considered authoritative on this policy, as we've frequently
had to tweak this to deal with out of disk issues.
### rootless containers
We are considering [podman](https://podman.io/) for running containers more securely:
