@@ -120,6 +120,19 @@ Alternatively, this will also clean old containers:
...
@@ -120,6 +120,19 @@ Alternatively, this will also clean old containers:
/usr/local/sbin/tpo-docker-clean-cache
/usr/local/sbin/tpo-docker-clean-cache
### DNS resolution failures
Under certain circumstances (upgrades?) Docker loses DNS resolution
(and possibly all of networking?). A symptom is that it simply fails
to clone the repository at the start of the job, for example:
fatal: unable to access 'https://gitlab-ci-token:[MASKED]@gitlab.torproject.org/tpo/network-health/sbws.git/': Could not resolve host: gitlab.torproject.org
A workaround is to reboot the runner's virtual machine. It might be
that we need to do some more configuration of Docker, see [upstream
issue 6644](https://gitlab.com/gitlab-org/gitlab-runner/-/issues/6644), although it's unclear why this problem is happening
right now. Still to be more fully, see [tpo/tpa/gitlab#93](https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/93).
## Disaster recovery
## Disaster recovery
Runners should be disposable: if a runner is destroyed, at most the
Runners should be disposable: if a runner is destroyed, at most the
