Changes

Page history
service/forum: switch to self-hosted documentation (#41063) authored by Jérôme Charaoui's avatar Jérôme Charaoui
Show whitespace changes
Inline Side-by-side
service/forum.md
View page @ c1df34cf
...@@ -4,14 +4,12 @@ The [Tor forum][] is currently hosted free of charge by Discourse.org for the ...@@ -4,14 +4,12 @@ The [Tor forum][] is currently hosted free of charge by Discourse.org for the
benefit of the Tor community. benefit of the Tor community.
[Discourse]: https://www.discourse.org/ [Discourse]: https://www.discourse.org/
[Tor forum]: https://forum.torproject.net/ [Tor forum]: https://forum.torproject.org/
[[_TOC_]] [[_TOC_]]
# Tutorial # Tutorial
# How-to
## Enable new topics by email ## Enable new topics by email
Topic creation by email is the ability to create a new forum topic in a category Topic creation by email is the ability to create a new forum topic in a category
...@@ -19,12 +17,10 @@ simply by sending an email to a specific address. ...@@ -19,12 +17,10 @@ simply by sending an email to a specific address.
This feature is enabled per-category. To enable it for a category, navigate to This feature is enabled per-category. To enable it for a category, navigate to
it, click the "wrench" icon (top right), open the **Settings** tab and scroll to it, click the "wrench" icon (top right), open the **Settings** tab and scroll to
the **Email** header. Check the box next to `Accept incoming emails sent to:`. the **Email** header.
The email address is managed automatically by Discourse, there's no need to Enter an email address under **Custom incoming email address**. The address to
create it. However, the address doesn't automatically appear on the forum so to use should be in the format `<categoryname>+discourse@forum.torproject.org`.
allow users to discover it, it's a good idea to add it to the category
description.
Per the forum's settings, only users with trust level 2 (member) or higher are Per the forum's settings, only users with trust level 2 (member) or higher are
allowed to post new topics by email. allowed to post new topics by email.
...@@ -44,160 +40,256 @@ The instructions to set up a forum category that mirrors for a mailing list can ...@@ -44,160 +40,256 @@ The instructions to set up a forum category that mirrors for a mailing list can
be found [here](https://meta.discourse.org/t/creating-a-read-only-mailing-list-mirror/77990#category-2). be found [here](https://meta.discourse.org/t/creating-a-read-only-mailing-list-mirror/77990#category-2).
The address that needs to be subscribed to the mailing list is The address that needs to be subscribed to the mailing list is
`torproject1@discoursemail.com`. `discourse@forum.torproject.org`.
# How-to
## Launch the Discourse Rails console
Log-in to the server's console as root and run:
cd /srv/discourse
./launcher enter app
rails c
## Reset a user's two-factor auth settings
In case a user can't log-in anymore due to two-factor authentication
parameters, it's possible to reset those using the Rails console.
First, load the user object by email, username or id:
user=User.find_by_email('email')
user=User.find_by_username('username')
user=User.find(id)
Then, simply run these two commands:
user.user_second_factors.destroy_all
user.security_keys.destroy_all
These instructions are copied from [this post](https://meta.discourse.org/t/disable-2fa-via-console/148309)
on the Discourse Meta forum.
## Reset a user account password
Usually when there is a need to reset a user's password, the user can
self-service through the forgotten password forum.
In case of issues with email, the password can also be reset from the Rails
console:
First, load the user object by email, username or id:
user=User.find_by_email('email')
user=User.find_by_username('username')
user=User.find(id)
Then:
user.password='passwordstring'
user.save!
These instructions are copied from [this post](https://meta.discourse.org/t/reset-user-password-by-admin/56987/7)
on the Discourse Meta forum.
## Adding or removing plugins
The plugins installed on our Discourse instance are configured using Puppet, in
`hiera/role/forum.yaml`.
To add or remove a plugin, simply add/remove the repository URL to the
`profile::discourse::plugins` key, run the Puppet agent and rebuild the
container:
./launcher rebuild app
This process can take a few minutes, during which the forum is unavailable.
Discourse has a plugins directory here: https://www.discourse.org/plugins
## Pager playbook ## Pager playbook
In case of a problem with the forum, we should start by checking the Discouse ### Un-delete a topic
[status page](https://status.discourse.org/) and the [@DiscourseStatus](https://twitter.com/DiscourseStatus) Twitter profile.
For any inquiries regarding issues with the forum, including unacknowledged As an admin user, the list of all deleted topics may be shown by navigating to
downtime, we should contact team@discourse.org immediately. https://forum.torproject.org/latest?status=deleted
## Disaster recovery Tu un-delete a topic, open it, click the wrench icon and select
`Un-delete topic`.
### Permanently destroy a topic
If a topic needs to be purged from Discourse, this can be accomplished using the
Rails console as follows, using the numeric topic identifier:
Topic.find(topic_id).destroy
These instructions are copied from [this post](https://meta.discourse.org/t/how-can-i-delete-or-undelete-topics/41700/9)
on the Discourse Meta forum.
In case of a major problem with the forum, we should contact the Discourse ### Enter the Discourse container
people at team@discourse.org. They have access to offsite backups.
Manual backups can be triggered and retrieved via the web interface, It's possible to enter the Discourse container to look around, make
see [Backups](#backups). Such backups could then be used to recreate the modifications, and restart the Discourse daemon itself.
forum on another installation of Discourse, in case of a catastrophe.
cd /srv/discourse
./launcher enter app
Any changes made in the container will be lost on upgrades, or when the
container is rebuilt using `./launcher rebuild app`.
Within the container its possible to restart the Discourse daemon using:
sv restart unicorn
## Disaster recovery
# Reference # Reference
## Installation ## Installation
The forum software was installed for us by the Discourse team, in their cloud Our installation is modeled after upstream's [recommended procedure] for
infrastructure, so the only thing that needed to be set up from our side was a deploying a single-server Docker-based instance of Discourse.
DNS entry.
The address chosen for this service is https://forum.torproject.net/. First, a [new machine] is required, with the following parameters:
It was decided to use the `.net` suffix instead of `.org` per [TPA-RFC-6](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-6-naming-convention). - an 80GB SSD-backed volume for container images and user uploads
- a 20GB NVMe-backed volume for the database
### Domain rediection Directories and mounts should be configured in the following manner:
For convenience, a HTTP redirection is set up in our infrastructure pointing - the SSD volume mounted on `/srv`
`forum.torproject.org` to `forum.torproject.net`. - `/srv/docker` bind mounted onto `/var/lib/docker`
### Onion service When this is ready, the `role::forum` Puppet class may be deployed onto the
machine. This will install Discourse's Docker manager software to
`/srv/discourse` along with the TPO-specific container templates for the main
application (`app.yml`) and the mail bridge (`mail-receiver.yml`).
We asked for this but unfortunately the Discourse team isn't interested in Once the catalog is applied, a few more steps are needed:
setting up an onion service for `forum.torproject.net`. This may be an argument
in favor of hosting the forum on our own infrastructure down the road.
They suggested we make one available via proxy, but we didn't look into that. 1. Bootstrap and start Discourse with these commands:
## SLA cd /srv/discourse
./launcher bootstrap app
./launcher start app
Since the forum is gratuitously hosted by Discourse for the Tor Project, the SLA 2. Login to https://forum.torproject.org and create a new admin account
on this service is "best effort".
The [status history](https://status.discourse.org/pages/history/5e2141ce30dc5c04b3ac32fc) 3. [Create an API key](#API-key-for-incoming-mail) using the instructions
suggests that outages are relatively rare. below
## Design 4. Run the Puppet agent on the machine to deploy the mail-receiver
### Akismet plugin [recommended procedure]: https://github.com/discourse/discourse/blob/main/docs/INSTALL-cloud.md
[new machine]: https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/new-machine
The Akismet plugin, enabled by default, is set to never submit data to Akismet ### API key for incoming mail
via the `skip akismet trust level` being set to `0:new user`.
This was discussed in a forum thread [here](https://forum.torproject.net/t/disable-akismet-anti-spam-plugin/80). Our Discourse setup relies on Postfix to transport incoming and outgoing mail,
such as notifications. For incoming mail, Postfix submits it to a
special `mail-receiver` container that is used to deliver email into Discourse
via its web API. A key is needed to authenticate the daemon running inside the
container.
## Issues To create and configure the API key:
There is no issue tracker specifically for this project, [File][] or 1. Login to Discourse using the administrator account
[search][] for issues in the [team issue tracker][search].
[File]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/new 2. Navigate to https://forum.torproject.org/admin/api/keys
[search]: https://gitlab.torproject.org/tpo/tpa/team/-/issues
## Maintainer, users, and upstream 3. Click the `New API Key` button
Upstream and service maintainers are [Discourse](mailto:team@discourse.org). 4. In the Description write `Incoming mail`, for User Level select `All Users`
and for Scope select `Granular`
This service is available publicly for the benefit of the entire Tor community. 5. Locate `email` under `topics` and check the box next to `receive emails`
The forum is administered by the service admins which are lavamind, hiro, gus 6. Click `Save`
and duncan.
## Monitoring and testing 7. Copy the generated key, then logon to the Puppet server run this command to
enter the API key into the database:
At present TPA does not monitor `forum.discourse.net` in any way. trocla set forum.torproject.org::discourse::mail_apikey plain
## Logs and metrics ## Onion service
Some logs and metrics are recorded by Discourse. Details are available in their An onion service is configured on the machine using Puppet, listening on ports
[privacy policy](https://www.discourse.org/privacy). 80 and 443.
## Backups Internally, Discourse has a `force_https` setting which determines whether
links are generated using the `http` or `https` scheme, and affects CSP URLs.
When this is enabled, the forum does not work using the onion service because
CSP URLs in the headers sent by Discourse are generated with the `https`
scheme. When the parameter is disabled, the main issue is that the links in
notifications all use the `http` scheme.
This service is hosted by Discourse.org which handle creating and storing So the most straightforward fix is simply to serve the forum via `https` on the
backups for the forum: onion service, that way we can leave the `force_https` setting enabled, and the
CSP headers don't prevent forum pages from loading.
> Off-site backups are created every 12 hours when hosted by Discourse. To access an off-site backup, or to restore an uploaded backup, please [contact us](mailto:team@discourse.org). ## Directory structure
It's possible to trigger the creation of a downloadable backup via this [page](https://forum.torproject.net/admin/backups) The purpose of the various directories under `/srv/discourse` is described in
on the admin interface. Once the backup is completed, a download link is sent by the [discourse_docker README](https://github.com/discourse/discourse_docker#directory-structure).
email to the admin who requested the backup.
## Deleting topics and posts The most important directories are:
The design of Discourse is such that when a post is deleted via the web - `containers`: contains our Docker container setup configurations
interface, it's merely flagged as such in the database. There is no purging of - `shared`: contains the logs, files and Postgresql database of the forum
these records either, they remain for the life of the instance.
To delete a post on Discourse requires direct modification of the database, which ## Design
is obviously warned against by the developpers.
## Other documentation
* https://meta.discourse.org/ ### Docker manager
# Discussion The Discourse Docker manager is installed under `/srv/discourse` and is
responsible for setting up the containers making up the Discourse installation.
## Overview The containers themselves are stateless, which means that they can be destroyed
and rebuilt without any data loss. All of the Discourse data is stored under
`/srv/discourse/shared`, including the Postgresql database.
<!-- describe the overall project. should include a link to a ticket --> ## Issues
<!-- that has a launch checklist -->
<!-- if this is an old project being documented, summarize the known --> There is no issue tracker specifically for this project, [File][] or
<!-- issues with the project. to quote the "audit procedure": [search][] for issues in the [team issue tracker][search].
5. When was the last security review done on the project? What was [File]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/new
the outcome? Are there any security issues currently? Should it [search]: https://gitlab.torproject.org/tpo/tpa/team/-/issues
have another security review?
6. When was the last risk assessment done? Something that would cover ## Maintainer, users, and upstream
risks from the data stored, the access required, etc.
7. Are there any in-progress projects? Technical debt cleanup? Upstream is [Discourse.org](https://www.discourse.org/).
Migrations? What state are they in? What's the urgency? What's the
next steps?
8. What urgent things need to be done on this project? This service is available publicly for the benefit of the entire Tor community.
--> The forum hosted on TPA infrastructure and administered by the service admins
which are lavamind, hiro, gus and duncan.
## Goals ## Monitoring and testing
<!-- include bugs to be fixed --> Only general Nagios/Prometheus monitoring is in place on the instance, there is
no Discourse-specific monitoring in place.
### Must have ## Logs and metrics
### Nice to have Logs for the main Discourse container (`app`) are located under
`/srv/discourse/shared/standalone/log`.
### Non-Goals The `mail-receiver` container logs can be consulted with:
## Approvals required cd /srv/discourse
./launcher logs mail-receiver
<!-- for example, legal, "vegas", accounting, current maintainer --> ## Backups
## Proposed Solution Backups containing the database and uploads are generated daily by Discourse
itself in `/srv/discourse/shared/standalone/backups`.
## Cost All other directories under `/srv/discourse/shared/standalone` are excluded from
Bacula backups configured from `/etc/bacula/local-exclude`.
## Alternatives considered ## Other documentation
<!-- include benchmarks and procedure if relevant --> * https://meta.discourse.org/