diff --git a/service/tor-weather.md b/service/tor-weather.md index cc6e431f2ec629f58e0af6683676a1d650f364d9..15226d08375df507bd1492dc23788687d664f896 100644 --- a/service/tor-weather.md +++ b/service/tor-weather.md @@ -88,16 +88,13 @@ SMTP_HOST=localhost SMTP_PORT=25 SMTP_USERNAME=weather@torproject.org SMTP_PASSWORD='' -SMTP_SSL='false' SQLALCHEMY_DATABASE_URI='postgresql+psycopg2://torweather:<database password>@localhost:5432/torweather' +BROKER_URL='amqp://torweather:<broker password>@localhost:5672' API_URL='https://onionoo.torproject.org' BASE_URL='https://weather.torproject.org' -# XXX: change this -JWT_SECRET='CHANGEME' - ONIONOO_JOB_INTERVAL=15 # XXX: change this @@ -106,6 +103,16 @@ EMAIL_ENCRYPT_PASS='Q0hBTkdFTUVDSEFOR0VNRUNIQU5HRU1FQ0hBTkdFTUU=' # XXX: change this SECRET_KEY='secret' + +SQLALCHEMY_TRACK_MODIFICATIONS= +CELERY_BIN=/home/weather/tor-weather-venv/bin/celery +CELERY_APP=tor_weather.celery.celery +CELERYD_NODES=worker1 +CELERYD_LOG_FILE=logs/celery/%n%I.log +CELERYD_LOG_LEVEL=info +CELERYD_OPTS= +CELERYBEAT_LOG_FILE=logs/celery/beat.log +CELERYBEAT_LOG_LEVEL=info ``` [Creating the postgres database]: <#creating-the-postgres-database> @@ -117,12 +124,14 @@ SECRET_KEY='secret' 1. activate the tor-weather virtualenv 1. install the wheel: `pip install tor_weather-0.1.0-py3-none-any.whl` 1. restart the service: `sudo -u weather env XDG_RUNTIME_DIR=/run/user/$(id -u weather) systemctl --user restart tor-weather.service` +1. restart the celery service: `sudo -u weather env XDG_RUNTIME_DIR=/run/user/$(id -u weather) systemctl --user restart tor-weather-celery.service` +1. restart the celery beat service: `sudo -u weather env XDG_RUNTIME_DIR=/run/user/$(id -u weather) systemctl --user restart tor-weather-celerybeat.service` [tor-weather ci]: <https://gitlab.torproject.org/tpo/network-health/tor-weather/-/pipelines> ## Migrating the database schema -After an upgrade or an initial deployment, you'll need to create or migrate the database schema. This script will activate the tor-weather virtual environment, export the tor-weather envvar settings, and then create/migrate the database schema. +After an upgrade or an initial deployment, you'll need to create or migrate the database schema. This script will activate the tor-weather virtual environment, export the tor-weather envvar settings, and then create/migrate the database schema. Note: the `flask` command might need to get updated dependent on the Python version running. ``` sudo -u weather bash @@ -131,7 +140,7 @@ source tor-weather-venv/bin/activate set -a source .tor-weather.env set +a -python -m tor_weather.data_model +flask --app tor_weather.app db upgrade --directory /home/weather/tor-weather-venv/lib/python3.11/site-packages/tor_weather/migrations exit ``` @@ -150,15 +159,21 @@ exit ## Services -the tor-weather deployment consists of three main services: +The tor-weather deployment consists of three main services: 1. apache: configured in puppet. proxies requests to `http://localhost:8000` 1. gunicorn: started by a systemd service file configured in puppet. runs with 5 workers (recommended by gunicorn docs: (2 * nproc) + 1), listens on localhost port 8000 1. postgres: a base postgres installation with a `torweather` user and database +Additionally, there are three services related to task scheduling: + +1. rabbitmq: configured in puppet, a message broker (listening on `localhost:5672`) +1. celery: task queue, started by a systemd service file configured in puppet +1. celery beat: scheduler, started by a systemd service file configured in puppet + ## Storage -tor-weather is backed by a postgres database. the postgres database is configured in the `/home/weather/tor-weather/env` file, using a [sqlalchemy connection URI][]. +tor-weather is backed by a postgres database. the postgres database is configured in the `/home/weather/.tor-weather.env` file, using a [sqlalchemy connection URI][]. [sqlalchemy connection URI]: <https://docs.sqlalchemy.org/en/20/core/engines.html#database-urls> @@ -166,7 +181,7 @@ tor-weather is backed by a postgres database. the postgres database is configure ### Onionoo Update Job -The `tor-weather-onionoo.service` and `tor-weather-onionoo.timer` files run a job every 15 minutes to update tor-weather's onionoo metrics information. Tor-weather handling its own job scheduling is a planned feature. +The `tor-weather-celerybeat.service` file triggers a job every 15 minutes to update tor-weather's onionoo metrics information. ## Interfaces