service/donate.md

+Donate-review is a GitLab runner and Review App deployment target for
+[donate-neo](gitlab.torproject.org/tpo/web/donate-neo).
+
+[[_TOC_]]
+
+# Tutorial
+
+## Starting a review app
+
+Pushing a commit on a non-main branch in the project repository will trigger a
+CI pipeline that includes `deploy-review` job. This job will deploy a review
+app hosted at `<branchname>.donate-review.torproject.net`.
+
+Commits to the `main` branch will be deployed to a review app by the
+`deploy-staging` job. The deployment process is similar except the app will be
+hosted at `staging.donate-review.torproject.net`.
+
+All review apps are automatically stopped and cleaned up once the associated
+branch is deleted.
+
+# How-to
+
+## Pager playbook
+
+# Disaster recovery
+
+In the event that the box is compromised, it should be rebuilt from scratch.
+See [Installation](#installation) below.
+
+# Reference
+
+## Installation
+
+1. bootstrap a new vm, and set up puppet
+1. add the `role: donate_review` parameter to the new machine in hiera-enc
+1. run puppet
+
+## Upgrades
+
+Upgrades are performed automatically through Debian packages. `gitlab-runner` is
+excluded from unattended-upgrades and must be upgraded manually.
+
+The review apps are not upgraded unless new commits appear in their branch to
+trigger a rebuild.
+
+TODO: The `renovate-cron` project should be enabled here to ensure timely
+upgrades to the staging and production deployments.
+
+## Design and architecture
+
+There are three major pieces to donate-review:
+
+- the donate-neo [.gitlab-ci.yml file][]
+- the `review-app.conf` apache2 configuration file
+- the `ci-reviewapp-generate-vhosts` script
+
+When a new feature branch is pushed to the project repository, the CI pipeline
+will build a new container and store it in the project's container registry.
+
+If tests are successful, the pipeline will then run a job on the shell executor
+to create (or update) a rootless podman container in the `gitlab-runner` user
+context. This container is set up to expose its internal port 8000 to a random
+outside port on the host.
+
+Finally, the `ci-reviewapp-generate-vhosts` script is executed via `sudo`. It
+will inspect all the running review app containers and create a configuration
+file where each line will instantiate a virtual host macro. These virtual hosts
+will proxy incoming connections to the appropriate port where the container is
+listening.
+
+[.gitlab-ci.yml file]: <https://gitlab.torproject.org/tpo/web/donate-neo/-/blob/main/.gitlab-ci.yml>
+
+### Subdomains and TLS Certs
+
+A wildcard certificate for `*.donate-review.torproject.net` is used for all
+review apps virtual host configurations.
+
+## Services
+
+- gitlab-runner
+- apache
+- podman containers (one per review app)
+
+## Authentication
+
+The donate-review *runner* uses token authentication to pick up jobs from
+GitLab. To access the review apps, HTTP basic authentication is required to
+prevent passers-by from stumbling onto the review apps and to keep indexing
+bots at bay. The username is `tor-www` and the password is blank.
+
+The Django-based review apps don't handle authentication, as there are no
+management users created by the app deployed from feature branches.
+
+The staging instance deployed from `main` does have a superuser with access to
+the management interface. Since the staging instance database is persistent,
+it's only necessary to create the user account once, manually. The command to
+do this is:
+
+    podman exec --interactive --tty donate-neo_main poetry run ./manage.py createsuperuser
+
+## Issues
+
+There is no issue tracker specifically for this the donate-review runner.
+[File][] or [search][] for issues in the donate-neo repository.
+
+ [File]: https://gitlab.torproject.org/tpo/web/donate-neo/-/issues/new
+ [search]: https://gitlab.torproject.org/tpo/web/donate-neo/-/issues
+
+## Maintainer
+
+[TPA](https://gitlab.torproject.org/tpo/tpa/team)
+
+## Users
+
+Anyone contributing to [tpo/web/donate-neo][].
+
+[tpo/web/donate-neo]: https://gitlab.torproject.org/tpo/web/donate-neo
+
+## Logs
+
+The review app logs may be accessed using the `podman logs <container>` command
+as the `gitlab-runner` user.
+
+## Backups
+
+This service has no special backup needs. All the donate-neo instances are
+ephemeral, and a new system can be bootstrapped solely from puppet.
+
+## Overview
+
+donate-review was created as part of tpo/web/donate-neo#6,
+tpo/tpa/team#41108 and refactored as part of tpo/web/donate-neo#21.
+
+Donate-review's purpose is to provide a review app deploy target for
+tpo/web/donate-neo. Most of the other tpo/web sites are static lektor sites,
+and can be easily deployed to a review app target as simple static sites
+fronted by Apache. But because donate-neo is a django application, it needs a
+specially-created deploy target for review apps.
+
+## Security and risk assessment
+
+donate-review is a shell executor, which means each CI job is executed with no
+real sandboxing or containerization. There was an attempt to set up the runner
+using systemd-nspawn, but it was taking too long and we
+[eventually decided against it][].
+
+Currently, project members with `Developper` permission or above in the
+donate-neo project may edit the CI configuration to execute arbitrary commands
+as the `gitlab-runner` user on the machine. Since these users are all trusted
+contributors, this should pose no problem. However, care should be taken to
+ensure no untrusted party is allowed to gain this privilege.
+
+[eventually decided against it]: <https://gitlab.torproject.org/tpo/tpa/team/-/issues/41108#note_2913481>
+
+## Technical debt and next steps
+
+The next step here is to make the donate-review service fully generic to allow
+other web projects with special runtime requirements to deploy review apps in
+the same manner.