document the Docker Hub registry mirror (#42181) authored by zen's avatar zen
......@@ -828,6 +828,42 @@ where that expired. Ideally, the ephemeral `CI_JOB_TOKEN` should be
usable for this, see [upstream gitlab-org/gitlab#438781](https://gitlab.com/gitlab-org/gitlab/-/issues/438781) for that
proposal.
### Docker Hub mirror
To workaround issues with [Docker Hub's pull rate limit][] (eg. #40335,
#42245), we deployed a container registry that acts as a read-only pull-through
proxy cache (#42181), effectively serving as a mirror of Docker Hub. All our
Docker GitLab Runners are automatically [configured to transparently pull from
the mirror][] when trying to fetch container images from the `docker.io`
namespace.
The service is available at https://dockerhub-mirror.torproject.org (initially
deployed at `dockerhub-mirror-01.torproject.org`) but only [Docker GitLab
Runners managed by TPA are allowed to connect][].
The service is managed via the [`role::registry_mirror`][] role and
[`profile::registry_mirror`][] profile and deploys:
- an Nginx frontend with a Let's Encrypt TLS certificate that listens on the
public addresses and acts as a reverse-proxy to the backend,
- a registry mirror backend that is provided by the [`docker-registry` package
in Debian][], and
- configuration for storing all registry data (i.e. image metadata and layers)
in the [MinIO object storage][].
The registry mirror [expires the cache after 7 days, by default][], and
[periodically removes old content to save disk space][].
[Docker Hub's pull rate limit]: https://docs.dgocker.com/docker-hub/usage/pulls/#view-pull-rate-and-limit
[configured to transparently pull from the mirror]: https://gitlab.torproject.org/tpo/tpa/puppet-control/-/blob/c6f9baae7b4ad5fe87a2df823a61711df626dd41/site/profile/manifests/gitlab/runner/docker.pp#L113
[Docker GitLab Runners managed by TPA are allowed to connect]: https://gitlab.torproject.org/tpo/tpa/puppet-control/-/blob/c6f9baae7b4ad5fe87a2df823a61711df626dd41/site/profile/manifests/registry_mirror.pp#L10
[`docker-registry` package in Debian]: https://tracker.debian.org/pkg/docker-registry
[MinIO object storage]: service/object-storate
[`role::registry_mirror`]: https://gitlab.torproject.org/tpo/tpa/puppet-control/-/blob/c6f9baae7b4ad5fe87a2df823a61711df626dd41/site/role/manifests/registry_mirror.pp
[`profile::registry_mirror`]: https://gitlab.torproject.org/tpo/tpa/puppet-control/-/blob/c6f9baae7b4ad5fe87a2df823a61711df626dd41/site/profile/manifests/registry_mirror.pp
[periodically removes old content to save disk space]: https://distribution.github.io/distribution/recipes/mirror/#what-about-my-disk
[expires the cache after 7 days, by default]: https://distribution.github.io/distribution/about/configuration/#proxy
## Issues
[File][] or [search][] for issues in our [GitLab issue
......
......