Skip to content

Changes

Page history
upload status-site design docs after gitlab migration authored by anarcat's avatar anarcat
Show whitespace changes
Inline Side-by-side
service/status.md
View page @ e884a50b
......@@ -15,7 +15,7 @@ have a copy of the [Hugo](https://gohugo.io/) static site generator
and the git repository:
sudo apt install hugo
git clone --recursive -b main git@git-rw.torproject.org:project/web/status-site
git clone --recursive -b main https://gitlab.torproject.org/tpo/tpa/status-site.git
cd status-site
WARNING: the URL of the Git repository changed! It used to be hosted
......@@ -60,39 +60,21 @@ before it is ready.
## Uploading site to the static mirror system
The `status.torproject.org` site currently lives in the [static mirror
system](howto/static-component) and uses the [git-rw repository](https://gitweb.torproject.org/project/web/status-site.git/) which get built via
Jenkins once you push to the repository.
In other words, uploading the site is automated by continuous
integration. So you simply need to commit and push:
Uploading the site is automated by continuous integration. So you
simply need to commit and push:
git commit -a -myolo
git push
Note that only the `webwml` group has access to the repository for
now.
You will see progress of the Jenkins jobs:
Note that only the `TPA` group has access to the repository for now,
but other users can request access as needed.
* [hugo-website-status](https://jenkins.torproject.org/job/hugo-website-status/) (build job)
* [hugo-website-status-install](https://jenkins.torproject.org/job/hugo-website-status-install/) (install job)
You can see the progress of build jobs in the [GitLab CI
pipelines](https://gitlab.torproject.org/tpo/tpa/status-site/-/pipelines). If all goes well, successful webhook deliveries should
show up in [this control panel](https://gitlab.torproject.org/tpo/tpa/status-site/-/hooks/1198/edit) as well.
If all goes well, the changes should propagate to the mirrors within
about 5 to 10 minutes, depending on how busy Jenkins is.
If the jobs did not trigger, make sure you are pushing to the Gitolite
server (`git-rw.torproject.org`) and *NOT* the GitLab server, which is
just a mirror and cannot currently trigger Jenkins jobs.
Merge requests may also be issued from the mirror of the repository on
GitLab:
<https://gitlab.torproject.org/tpo/tpa/status-site>
... but will need to be merged into the `git-rw` server by someone in
the above group to take effect. More people have access to the GitLab
repository and should therefore be able to collaborate there.
a few seconds to a minute.
See also the [disaster recovery](#disaster-recovery) options below.
......@@ -137,26 +119,23 @@ Those procedures were not tested.
### Manual deployment to the static mirror system
If `git-rw` is down, you can upload the `public/` folder content under
`/srv/status.torproject.org/htdocs`.
If GitLab is down, you can upload the `public/` folder content under
`/srv/static-gitlab-shim/status.torproject.org/`.
The canonical source for the static websites rotation is defined in
Puppet (in `modules/roles/misc/static-components.yaml`) and is
currently set to `staticiforme.torproject.org`. This `rsync` command
currently set to `static-gitlab-shim.torproject.org`. This `rsync` command
should be enough:
rsync -rtP public/ anarcat@staticiforme.torproject.org:/srv/status.torproject.org/htdocs
NOTE: there is a copy of the git repository in `/status-site` as
well. Ignore it: it's out of date but could be used to build the
website in a pinch.
rsync -rtP public/ anarcat@staticiforme.torproject.org:/srv/static-gitlab-shim/status.torproject.org
Then the new source material needs to be synchronized to the mirrors,
with:
sudo -u torwww static-update-component status.torproject.org
sudo -u mirroradm static-update-component status.torproject.org
This requires membership to the `torwww` group.
This requires access to the `mirroradm` group, although typically the
machine is only accessible to TPA anyways.
Don't forget to push the changes to the git repository, once that is
available. It's important so that the next people can start from your
......@@ -190,9 +169,6 @@ Then DNS needs to be tweaked to point there as well.
# Reference
The `status.tpo` dashboard is built with [cstate](https://github.com/cstate/cstate/), which is a theme
for the [Hugo](https://gohugo.io/) static site generator.
## Installation
See the instructions on how to [setup a local development
......@@ -208,25 +184,14 @@ deploy it to a third-party provider.
## Design
The status site is part of the [static mirror system](howto/static-component) and is built
with Jenkins jobs, from a git repository on the [git](howto/git) server. This
was setup this way because that is how every other static website is
currently built.
with [cstate](https://github.com/cstate/cstate/), which is a theme for the [Hugo](https://gohugo.io/) static site
generator. The site is managed in a git repository on the [GitLab](howto/gitlab)
server and uses [GitLab CI](service/ci) to get built. The
[static-gitlab-shim service](service/static-gitlab-shim) propagates the builds to the [static
mirror system](howto/static-component) for high availability.
This involved:
* a new [static component](howto/static-component) owned by `torwww` (in the
`tor-puppet.git` repository)
* a new [build script](https://gitweb.torproject.org/project/jenkins/tools.git/tree/slaves/linux/hugo-website-status?id=7fff67041e41206fef7d128ac60488db375652a1) in the [jenkins/tools.git](https://gitweb.torproject.org/project/jenkins/tools.git/) repository
* a new [build job](https://gitweb.torproject.org/project/jenkins/jobs.git/commit/?id=170fd9879ff0da65670aaa36c20e63c0db1ed039) in the [jenkins/jobs.git](https://gitweb.torproject.org/project/jenkins/jobs.git/) repository
* a [new entry](https://gitweb.torproject.org/admin/static-builds.git/commit/?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the [ssh wrapper](https://gitweb.torproject.org/admin/static-builds.git/tree/ssh-wrap?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the
[admin/static-builds.git](https://gitweb.torproject.org/admin/static-builds.git/) repository
* a new [gitolite](howto/git) repository with hooks to ping the Jenkins server and
mirror to GitLab
We also considered using GitLab CI for deployment but (a) GitLab pages
is not yet setup and (b) it doesn't integrate well with the static
mirror system for now. See [the broader discussion of the static site
system improvements](howto/static-component#proposed-solution).
See the [static-gitlab-shim service design document](service/static-gitlab-shim#design) for more
information.
## Issues
......@@ -413,10 +378,37 @@ experience:
* [Wikimedia](http://status.wikimedia.org/) - based on [proprietary nimsoft software](http://www.nimsoft.com/solutions/nimsoft-cloud-user-experience/key-features/public-status-page.html?m=41159&c=pspfoot),
deprecated in favor of Grafana
## Previous implementation
## Previous implementations
### IRC bot
A similar service was ran by @weasel around 2014. It would bridge the
`status` comments on IRC into a website, see [this archived
version](https://web.archive.org/web/20141017061031/https://gs.torproject.org/)
and [the source code](https://github.com/weaselp/groupstatus), which
is still available.
### Jenkins jobs
The site used to be built with Jenkins jobs, from a git repository on
the [git](howto/git) server. This was setup this way because that is how every
other static website was built back then.
This involved:
* a new [static component](howto/static-component) owned by `torwww` (in the
`tor-puppet.git` repository)
* a new [build script](https://gitweb.torproject.org/project/jenkins/tools.git/tree/slaves/linux/hugo-website-status?id=7fff67041e41206fef7d128ac60488db375652a1) in the [jenkins/tools.git](https://gitweb.torproject.org/project/jenkins/tools.git/) repository
* a new [build job](https://gitweb.torproject.org/project/jenkins/jobs.git/commit/?id=170fd9879ff0da65670aaa36c20e63c0db1ed039) in the [jenkins/jobs.git](https://gitweb.torproject.org/project/jenkins/jobs.git/) repository
* a [new entry](https://gitweb.torproject.org/admin/static-builds.git/commit/?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the [ssh wrapper](https://gitweb.torproject.org/admin/static-builds.git/tree/ssh-wrap?id=b2344aa1d68f4f065764c6f23d14494020b81f86) in the
[admin/static-builds.git](https://gitweb.torproject.org/admin/static-builds.git/) repository
* a new [gitolite](howto/git) repository with hooks to ping the Jenkins server and
mirror to GitLab
We also considered using GitLab CI for deployment but (a) GitLab pages
was not yet setup and (b) it didn't integrate well with the static
mirror system for now. See [the broader discussion of the static site
system improvements](howto/static-component#proposed-solution).
Both issues have now been fixed thanks to the [static-gitlab-shim
service](service/static-gitlab-shim).