@@ -377,6 +377,10 @@ file. (We have considered taking this out of the template and writing
a proper Python script, but then users would have to copy that script
over their repo, or clone a repo in CI, and that seems impractical.)
Another thing we considered is to set [instance-level templates](https://docs.gitlab.com/ee/user/admin_area/settings/instance_template_repository.html)
but it seems that feature is not available in GitLab's free software
version.
The CI hooks are deployed by users, which will typically include the
above template in their own `.gitlab-ci.yml` file.
...
...
@@ -476,6 +480,7 @@ good copies of the websites, in any case.
## Other documentation
*[TPA-RFC-10][]: Jenkins retirement
* GitLab's [CI deployment mechanism](https://about.gitlab.com/blog/2021/02/05/ci-deployment-and-environments/) blog post
*[Design and launch ticket](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40364)
* our [static mirror system][] documentation
...
...
@@ -493,13 +498,15 @@ good copies of the websites, in any case.
## Overview
The static shim was built to unblock the [Jenkins retirement
project](https://gitlab.torproject.org/groups/tpo/-/milestones/27). A key blocker was that the [static mirror system][] was
strongly coupled with Jenkins: many high traffic and critical websites
project](https://gitlab.torproject.org/groups/tpo/-/milestones/27)([TPA-RFC-10][]). A key blocker was that the [static mirror system][] was
strongly coupled with Jenkins: many high traffic and [critical websites](policy/tpa-rfc-10-jenkins-retirement#critical-website-builds)
are built and deployed by Jenkins. Unless we wanted to completely
retire the static mirror system (in favor, say, of GitLab Pages), we
had to create a way for GitLab CI to deploy content to the static
mirror system.
[TPA-RFC-10]:policy/tpa-rfc-10-jenkins-retirement
This section contains more in-depth discussions about the reasoning
behind the project, discarded alternatives, and other ideas.
...
...
@@ -528,7 +535,7 @@ deployment.
## Approvals required
TPA.
TPA, part of [TPA-RFC-10][]: Jenkins retirement.
## Proposed Solution
...
...
@@ -628,7 +635,7 @@ created very frequently, it could also be done by hand.
Unfortunately this design has two major flaws:
1. webhooks are designed to be fast and short-lived: most site
1. webhooks are [designed to be fast and short-lived](https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#webhook-endpoint-tips): most site
deployments take longer than the pre-configured webhook timeout (10
seconds) and therefore cannot be deployed synchronously, which
implies that...
...
...
@@ -644,5 +651,26 @@ Unfortunately this design has two major flaws:
recent webhook calls, in Settings -> Webhooks -> Edit ->
Recent deliveries. But that is rather well-hidden.
In the short term, the webhook system has be used asynchronously,
but we have since moved to the deployment system documented above.
Note that it may have been possible to change the 10 seconds timeout
with:
gitlab_rails['webhook_timeout'] = 10
in the `/etc/gitlab/gitlab.rb` file ([source](https://docs.gitlab.com/ee/user/project/integrations/webhooks.html#webhook-fails-or-multiple-webhook-requests-are-triggered)). But static site
deployments can take a while, so it's not clear at all we can actually
wait for the full deployment.
In the short term, the webhook system has be used asynchronously (by
removing the `include-command-output-in-response` parameter in the
webhook config), but then the error reporting is even worse because
the caller doesn't even know if the deploy succeeds or fails.
We have since moved to the deployment system documented in the [design
section](#design).
### GitLab "Integrations"
Another approach we briefly considered is to write an [integration](https://docs.gitlab.com/ee/user/project/integrations/overview.html)
into GitLab. We found the documentation for this was nearly
nonexistent. It also meant maintaining a bundle of Ruby code inside
