service/forum: switch to self-hosted documentation (#41063) authored by Jérôme Charaoui's avatar Jérôme Charaoui
......@@ -4,14 +4,12 @@ The [Tor forum][] is currently hosted free of charge by Discourse.org for the
benefit of the Tor community.
[Discourse]: https://www.discourse.org/
[Tor forum]: https://forum.torproject.net/
[Tor forum]: https://forum.torproject.org/
[[_TOC_]]
# Tutorial
# How-to
## Enable new topics by email
Topic creation by email is the ability to create a new forum topic in a category
......@@ -19,12 +17,10 @@ simply by sending an email to a specific address.
This feature is enabled per-category. To enable it for a category, navigate to
it, click the "wrench" icon (top right), open the **Settings** tab and scroll to
the **Email** header. Check the box next to `Accept incoming emails sent to:`.
the **Email** header.
The email address is managed automatically by Discourse, there's no need to
create it. However, the address doesn't automatically appear on the forum so to
allow users to discover it, it's a good idea to add it to the category
description.
Enter an email address under **Custom incoming email address**. The address to
use should be in the format `<categoryname>+discourse@forum.torproject.org`.
Per the forum's settings, only users with trust level 2 (member) or higher are
allowed to post new topics by email.
......@@ -44,160 +40,256 @@ The instructions to set up a forum category that mirrors for a mailing list can
be found [here](https://meta.discourse.org/t/creating-a-read-only-mailing-list-mirror/77990#category-2).
The address that needs to be subscribed to the mailing list is
`torproject1@discoursemail.com`.
`discourse@forum.torproject.org`.
# How-to
## Launch the Discourse Rails console
Log-in to the server's console as root and run:
cd /srv/discourse
./launcher enter app
rails c
## Reset a user's two-factor auth settings
In case a user can't log-in anymore due to two-factor authentication
parameters, it's possible to reset those using the Rails console.
First, load the user object by email, username or id:
user=User.find_by_email('email')
user=User.find_by_username('username')
user=User.find(id)
Then, simply run these two commands:
user.user_second_factors.destroy_all
user.security_keys.destroy_all
These instructions are copied from [this post](https://meta.discourse.org/t/disable-2fa-via-console/148309)
on the Discourse Meta forum.
## Reset a user account password
Usually when there is a need to reset a user's password, the user can
self-service through the forgotten password forum.
In case of issues with email, the password can also be reset from the Rails
console:
First, load the user object by email, username or id:
user=User.find_by_email('email')
user=User.find_by_username('username')
user=User.find(id)
Then:
user.password='passwordstring'
user.save!
These instructions are copied from [this post](https://meta.discourse.org/t/reset-user-password-by-admin/56987/7)
on the Discourse Meta forum.
## Adding or removing plugins
The plugins installed on our Discourse instance are configured using Puppet, in
`hiera/role/forum.yaml`.
To add or remove a plugin, simply add/remove the repository URL to the
`profile::discourse::plugins` key, run the Puppet agent and rebuild the
container:
./launcher rebuild app
This process can take a few minutes, during which the forum is unavailable.
Discourse has a plugins directory here: https://www.discourse.org/plugins
## Pager playbook
In case of a problem with the forum, we should start by checking the Discouse
[status page](https://status.discourse.org/) and the [@DiscourseStatus](https://twitter.com/DiscourseStatus) Twitter profile.
### Un-delete a topic
For any inquiries regarding issues with the forum, including unacknowledged
downtime, we should contact team@discourse.org immediately.
As an admin user, the list of all deleted topics may be shown by navigating to
https://forum.torproject.org/latest?status=deleted
## Disaster recovery
Tu un-delete a topic, open it, click the wrench icon and select
`Un-delete topic`.
### Permanently destroy a topic
If a topic needs to be purged from Discourse, this can be accomplished using the
Rails console as follows, using the numeric topic identifier:
Topic.find(topic_id).destroy
These instructions are copied from [this post](https://meta.discourse.org/t/how-can-i-delete-or-undelete-topics/41700/9)
on the Discourse Meta forum.
In case of a major problem with the forum, we should contact the Discourse
people at team@discourse.org. They have access to offsite backups.
### Enter the Discourse container
Manual backups can be triggered and retrieved via the web interface,
see [Backups](#backups). Such backups could then be used to recreate the
forum on another installation of Discourse, in case of a catastrophe.
It's possible to enter the Discourse container to look around, make
modifications, and restart the Discourse daemon itself.
cd /srv/discourse
./launcher enter app
Any changes made in the container will be lost on upgrades, or when the
container is rebuilt using `./launcher rebuild app`.
Within the container its possible to restart the Discourse daemon using:
sv restart unicorn
## Disaster recovery
# Reference
## Installation
The forum software was installed for us by the Discourse team, in their cloud
infrastructure, so the only thing that needed to be set up from our side was a
DNS entry.
Our installation is modeled after upstream's [recommended procedure] for
deploying a single-server Docker-based instance of Discourse.
The address chosen for this service is https://forum.torproject.net/.
First, a [new machine] is required, with the following parameters:
It was decided to use the `.net` suffix instead of `.org` per [TPA-RFC-6](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-6-naming-convention).
- an 80GB SSD-backed volume for container images and user uploads
- a 20GB NVMe-backed volume for the database
### Domain rediection
Directories and mounts should be configured in the following manner:
For convenience, a HTTP redirection is set up in our infrastructure pointing
`forum.torproject.org` to `forum.torproject.net`.
- the SSD volume mounted on `/srv`
- `/srv/docker` bind mounted onto `/var/lib/docker`
### Onion service
When this is ready, the `role::forum` Puppet class may be deployed onto the
machine. This will install Discourse's Docker manager software to
`/srv/discourse` along with the TPO-specific container templates for the main
application (`app.yml`) and the mail bridge (`mail-receiver.yml`).
We asked for this but unfortunately the Discourse team isn't interested in
setting up an onion service for `forum.torproject.net`. This may be an argument
in favor of hosting the forum on our own infrastructure down the road.
Once the catalog is applied, a few more steps are needed:
They suggested we make one available via proxy, but we didn't look into that.
1. Bootstrap and start Discourse with these commands:
## SLA
cd /srv/discourse
./launcher bootstrap app
./launcher start app
Since the forum is gratuitously hosted by Discourse for the Tor Project, the SLA
on this service is "best effort".
2. Login to https://forum.torproject.org and create a new admin account
The [status history](https://status.discourse.org/pages/history/5e2141ce30dc5c04b3ac32fc)
suggests that outages are relatively rare.
3. [Create an API key](#API-key-for-incoming-mail) using the instructions
below
## Design
4. Run the Puppet agent on the machine to deploy the mail-receiver
### Akismet plugin
[recommended procedure]: https://github.com/discourse/discourse/blob/main/docs/INSTALL-cloud.md
[new machine]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/new-machine
The Akismet plugin, enabled by default, is set to never submit data to Akismet
via the `skip akismet trust level` being set to `0:new user`.
### API key for incoming mail
This was discussed in a forum thread [here](https://forum.torproject.net/t/disable-akismet-anti-spam-plugin/80).
Our Discourse setup relies on Postfix to transport incoming and outgoing mail,
such as notifications. For incoming mail, Postfix submits it to a
special `mail-receiver` container that is used to deliver email into Discourse
via its web API. A key is needed to authenticate the daemon running inside the
container.
## Issues
To create and configure the API key:
There is no issue tracker specifically for this project, [File][] or
[search][] for issues in the [team issue tracker][search].
1. Login to Discourse using the administrator account
[File]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/new
[search]: https://gitlab.torproject.org/tpo/tpa/team/-/issues
2. Navigate to https://forum.torproject.org/admin/api/keys
## Maintainer, users, and upstream
3. Click the `New API Key` button
Upstream and service maintainers are [Discourse](mailto:team@discourse.org).
4. In the Description write `Incoming mail`, for User Level select `All Users`
and for Scope select `Granular`
This service is available publicly for the benefit of the entire Tor community.
5. Locate `email` under `topics` and check the box next to `receive emails`
The forum is administered by the service admins which are lavamind, hiro, gus
and duncan.
6. Click `Save`
## Monitoring and testing
7. Copy the generated key, then logon to the Puppet server run this command to
enter the API key into the database:
At present TPA does not monitor `forum.discourse.net` in any way.
trocla set forum.torproject.org::discourse::mail_apikey plain
## Logs and metrics
## Onion service
Some logs and metrics are recorded by Discourse. Details are available in their
[privacy policy](https://www.discourse.org/privacy).
An onion service is configured on the machine using Puppet, listening on ports
80 and 443.
## Backups
Internally, Discourse has a `force_https` setting which determines whether
links are generated using the `http` or `https` scheme, and affects CSP URLs.
When this is enabled, the forum does not work using the onion service because
CSP URLs in the headers sent by Discourse are generated with the `https`
scheme. When the parameter is disabled, the main issue is that the links in
notifications all use the `http` scheme.
This service is hosted by Discourse.org which handle creating and storing
backups for the forum:
So the most straightforward fix is simply to serve the forum via `https` on the
onion service, that way we can leave the `force_https` setting enabled, and the
CSP headers don't prevent forum pages from loading.
> Off-site backups are created every 12 hours when hosted by Discourse. To access an off-site backup, or to restore an uploaded backup, please [contact us](mailto:team@discourse.org).
## Directory structure
It's possible to trigger the creation of a downloadable backup via this [page](https://forum.torproject.net/admin/backups)
on the admin interface. Once the backup is completed, a download link is sent by
email to the admin who requested the backup.
The purpose of the various directories under `/srv/discourse` is described in
the [discourse_docker README](https://github.com/discourse/discourse_docker#directory-structure).
## Deleting topics and posts
The most important directories are:
The design of Discourse is such that when a post is deleted via the web
interface, it's merely flagged as such in the database. There is no purging of
these records either, they remain for the life of the instance.
- `containers`: contains our Docker container setup configurations
- `shared`: contains the logs, files and Postgresql database of the forum
To delete a post on Discourse requires direct modification of the database, which
is obviously warned against by the developpers.
## Other documentation
## Design
* https://meta.discourse.org/
### Docker manager
# Discussion
The Discourse Docker manager is installed under `/srv/discourse` and is
responsible for setting up the containers making up the Discourse installation.
## Overview
The containers themselves are stateless, which means that they can be destroyed
and rebuilt without any data loss. All of the Discourse data is stored under
`/srv/discourse/shared`, including the Postgresql database.
<!-- describe the overall project. should include a link to a ticket -->
<!-- that has a launch checklist -->
## Issues
<!-- if this is an old project being documented, summarize the known -->
<!-- issues with the project. to quote the "audit procedure":
There is no issue tracker specifically for this project, [File][] or
[search][] for issues in the [team issue tracker][search].
5. When was the last security review done on the project? What was
the outcome? Are there any security issues currently? Should it
have another security review?
[File]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/new
[search]: https://gitlab.torproject.org/tpo/tpa/team/-/issues
6. When was the last risk assessment done? Something that would cover
risks from the data stored, the access required, etc.
## Maintainer, users, and upstream
7. Are there any in-progress projects? Technical debt cleanup?
Migrations? What state are they in? What's the urgency? What's the
next steps?
Upstream is [Discourse.org](https://www.discourse.org/).
8. What urgent things need to be done on this project?
This service is available publicly for the benefit of the entire Tor community.
-->
The forum hosted on TPA infrastructure and administered by the service admins
which are lavamind, hiro, gus and duncan.
## Goals
## Monitoring and testing
<!-- include bugs to be fixed -->
Only general Nagios/Prometheus monitoring is in place on the instance, there is
no Discourse-specific monitoring in place.
### Must have
## Logs and metrics
### Nice to have
Logs for the main Discourse container (`app`) are located under
`/srv/discourse/shared/standalone/log`.
### Non-Goals
The `mail-receiver` container logs can be consulted with:
## Approvals required
cd /srv/discourse
./launcher logs mail-receiver
<!-- for example, legal, "vegas", accounting, current maintainer -->
## Backups
## Proposed Solution
Backups containing the database and uploads are generated daily by Discourse
itself in `/srv/discourse/shared/standalone/backups`.
## Cost
All other directories under `/srv/discourse/shared/standalone` are excluded from
Bacula backups configured from `/etc/bacula/local-exclude`.
## Alternatives considered
## Other documentation
<!-- include benchmarks and procedure if relevant -->
* https://meta.discourse.org/