... | ... | @@ -183,8 +183,8 @@ In this case, it was because of a prolonged outage on that host, which |
|
|
made it unreachable to the master server ([tpo/tpa/team#40432](https://gitlab.torproject.org/tpo/tpa/team/-/issues/incident/40432)).
|
|
|
|
|
|
The solution is to run a manual sync. This can be done by, for
|
|
|
example, pushing to Jenkins or running `static-update-component` by
|
|
|
hand, see [doc/static-sites](doc/static-sites).
|
|
|
example, running a deploy job in GitLab (see [static-shim](service/static-shim)) or
|
|
|
running `static-update-component` by hand, see [doc/static-sites](doc/static-sites).
|
|
|
|
|
|
In this particular case, the solution is simply to run this on the
|
|
|
static source (`palmeri` at the time of writing):
|
... | ... | @@ -374,6 +374,8 @@ hosts. Those components are defined in a YAML file in the |
|
|
(`modules/roles/misc/static-components.yaml` at the time of writing,
|
|
|
but it might move to Hiera, see [issue 30020](https://gitlab.torproject.org/tpo/tpa/team/-/issues/30020) and [puppet](puppet)).
|
|
|
|
|
|
TODO: update to point to the shim.
|
|
|
|
|
|
The Jenkins server is also used to build and push websites to static
|
|
|
source servers.
|
|
|
|
... | ... | @@ -558,11 +560,17 @@ The user's `sudo` configuration is therefore critical and that |
|
|
`sudoers` configuration could also be considered part of the static
|
|
|
mirror system.
|
|
|
|
|
|
Jenkins has SSH access to the `torwww` user in the static
|
|
|
infrastructure, so it can build and push websites, see below.
|
|
|
The GitLab runners have SSH access to the [static-shim](service/static-shim) service
|
|
|
infrastructure, so it can build and push websites, through a private
|
|
|
key kept in the project, the public part of which is deployed by
|
|
|
Puppet.
|
|
|
|
|
|
### Jenkins build jobs
|
|
|
|
|
|
WARNING: Jenkins was [retired in late 2021](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40218). This documentation is
|
|
|
now irrelevant and is kept only for historical purposes. The
|
|
|
[static-shim](service/static-shim) with [GitLab CI](service/ci) has replaced this.
|
|
|
|
|
|
Jenkins is used to build some websites and push them to the static
|
|
|
mirror infrastructure. The Jenkins jobs get triggered from `git-rw`
|
|
|
git hooks, and are (partially) defined in [jenkins/tools.git](https://gitweb.torproject.org/project/jenkins/tools.git/) and
|
... | ... | @@ -684,18 +692,6 @@ track. There has been some refactoring to move most of the code in a |
|
|
`staticsync` module, but we still have files strewn over other
|
|
|
modules.
|
|
|
|
|
|
For certain sites, the static site system requires Jenkins to build
|
|
|
websites, which further complicates deployments. A static site
|
|
|
deployment requiring Jenkins needs updates on 5 different
|
|
|
repositories, across 4 different services:
|
|
|
|
|
|
* a new static component in the (private) `tor-puppet.git` repository
|
|
|
* a [build script](https://gitweb.torproject.org/project/jenkins/tools.git/tree/slaves/linux/) in the [jenkins/tools.git](https://gitweb.torproject.org/project/jenkins/tools.git/) repository
|
|
|
* a build job in the [jenkins/jobs.git](https://gitweb.torproject.org/project/jenkins/jobs.git/) repository
|
|
|
* a [new entry](https://gitweb.torproject.org/admin/static-builds.git/commit/?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the [ssh wrapper](https://gitweb.torproject.org/admin/static-builds.git/tree/ssh-wrap?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the
|
|
|
[admin/static-builds.git](https://gitweb.torproject.org/admin/static-builds.git/) repository
|
|
|
* a new entry in the `gitolite-admin.git` repository
|
|
|
|
|
|
The static site system has no unit tests, linting, release process, or
|
|
|
CI. Code is deployed directly through Puppet, on the live servers.
|
|
|
|
... | ... | @@ -709,6 +705,25 @@ code. Thankfully it is fairly short and should be easy to port. |
|
|
The YAML configuration duplicates the YAML parsing and data structures
|
|
|
present in Hiera, see [issue 30020](https://gitlab.torproject.org/tpo/tpa/team/-/issues/30020) and [puppet](puppet)).
|
|
|
|
|
|
### Jenkins integration
|
|
|
|
|
|
NOTE: this section is now irrelevant, because Jenkins was retired in
|
|
|
favor of the [static-shim](service/static-shim) to [GitLab CI](service/ci). A new site now
|
|
|
requires only a change in GitLab and Puppet, successfully reducing
|
|
|
this list to 2 services and 2 repositories.
|
|
|
|
|
|
For certain sites, the static site system requires Jenkins to build
|
|
|
websites, which further complicates deployments. A static site
|
|
|
deployment requiring Jenkins needs updates on 5 different
|
|
|
repositories, across 4 different services:
|
|
|
|
|
|
* a new static component in the (private) `tor-puppet.git` repository
|
|
|
* a [build script](https://gitweb.torproject.org/project/jenkins/tools.git/tree/slaves/linux/) in the [jenkins/tools.git](https://gitweb.torproject.org/project/jenkins/tools.git/) repository
|
|
|
* a build job in the [jenkins/jobs.git](https://gitweb.torproject.org/project/jenkins/jobs.git/) repository
|
|
|
* a [new entry](https://gitweb.torproject.org/admin/static-builds.git/commit/?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the [ssh wrapper](https://gitweb.torproject.org/admin/static-builds.git/tree/ssh-wrap?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the
|
|
|
[admin/static-builds.git](https://gitweb.torproject.org/admin/static-builds.git/) repository
|
|
|
* a new entry in the `gitolite-admin.git` repository
|
|
|
|
|
|
## Goals
|
|
|
|
|
|
### Must have
|
... | ... | |