document jenkins / static-sites integration and alternatives

howto/static-component.md

@@ -374,6 +374,32 @@ The user's `sudo` configuration is therefore critical and that
 `sudoers` configuration could also be considered part of the static
 mirror system.
 
+Jenkins has SSH access to the `torwww` user in the static
+infrastructure, so it can build and push websites, see below.
+
+### Jenkins build jobs
+
+Jenkins is used to build some websites and push them to the static
+mirror infrastructure. The Jenkins jobs get triggered from `git-rw`
+git hooks, and are (partially) defined in [jenkins/tools.git](https://gitweb.torproject.org/project/jenkins/tools.git/) and
+[jenkins/jobs.git](https://gitweb.torproject.org/project/jenkins/jobs.git/). Those are fed into [jenkins-job-builder](https://docs.openstack.org/infra/jenkins-job-builder/) to
+build the actual job. Those jobs actually build the site with hugo or
+lektor and package an archive that is then fetched by the static
+source.
+
+The [build scripts](https://gitweb.torproject.org/admin/static-builds.git/) are deployed on `staticiforme`, in the
+`~torwww` home directory. Those get triggered through the
+`~torwww/bin/ssh-wrap` program, hardcoded in
+`/etc/ssh/userkeys/torwww`, which picks the right build job based on
+the argument provided by the Jenkins job, for example:
+
+        - shell: "cat incoming/output.tar.gz | ssh torwww@staticiforme.torproject.org hugo-website-{site}"
+
+Then the wrapper eventually does something like this to update the
+static component on the static source:
+
+    rsync --delete -v -r "${tmpdir}/incoming/output/." "${basedir}"
+    static-update-component "$component"
 
 ## Issues
 
@@ -508,6 +534,8 @@ powerful and provides a reasonably easy to deploy, high availability
 service, at the cost of some level of obscurity, complexity, and high
 disk space requirements.
 
+### Replacing with GitLab and cache
+
 It should be possible to replace parts or the entirety of the system
 progressively, however. A few ideas:
 
@@ -539,6 +567,34 @@ Next steps:
     scalable it actually is or if we'll need to run the cache frontend
     in front of it
 
+### Replacing Jenkins with GitLab CI as a builder
+
+We currently use Jenkins to build some websites and push them to the
+static mirror infrastructure, as documented above. To use GitLab CI
+here, there are a few alternatives.
+
+ 1. trigger Jenkins jobs from GitLab CI: there is [a GitLab plugin to
+    trigger Jenkins jobs](https://docs.gitlab.com/ee/integration/jenkins.html), but that doesn't actually replace
+    Jenkins
+ 2. replace Jenkins by replicating the `ssh` pipeline: this involves
+    shipping the private SSH key as a [private environment
+    variable](https://gitlab.torproject.org/help/ci/variables/README#custom-environment-variables) which then is used by the runner to send the file and
+    trigger the build. this is seen as a too broad security issue
+ 3. replace Jenkins by a web hook
+
+The web hook, in particular, would run on "jobs" changes, and would
+perform the following:
+
+ 1. run as a (Python? [WSGI](https://docs.python.org/3/library/wsgiref.html#module-wsgiref.simple_server)?) web server (wrapped by Apache?)
+ 2. listen to [webhooks from GitLab](https://gitlab.torproject.org/help/user/project/integrations/webhooks#pipeline-events), and only GitLab (ip allow list, in Apache?)
+ 3. map given project to given static site component (or secret token?)
+ 4. pull artifacts from job (do the equivalent to `wget` and `unzip`)
+ 5. `rsync -c` into local static source (to avoid reseting timestamps)
+ 6. triggers `static-update-component`
+
+This would mean a new service, but would allow us to retire Jenkins
+without rearchitecturing the entire static mirroring system.
+
 ## Cost
 
 Staff, mostly. We expect a reduction in cost if we reduce the number