longterm/setup.md

+# Instructions for setting up long-term instances
+
+  - [Environment setup](#environment-setup-)
+    - [Iptables setup](#iptables-setup-)
+    - [Hostname, utilities, and user setup](#hostname,-utilities,-and-user-setup-)
+  - [Compile Tor from source](#compile-tor-from-source-)
+  - [Compile TGen from source](#compile-tgen-from-source-)
+  - [Install OnionPerf](#install-onionperf-)
+  - [LetsEncrypt/Apache Installation](#letsencrypt/apache-installation-)
+    - [Add new htdocs/ URLs to CollecTor](#add-new-htdocs/-urls-to-collector-)
+  - [Monit Installation](#monit-installation-)
+    - [Greenhost initial instructions](#-)
+    - [Common Instructions for Greenhost/AWS](#common-instructions-for-greenhost/aws-)
+
+## Environment setup
+
+For running OnionPerf instances, we consider AWS or Greenhost VMs, with the following setup:
+
+   * Greenhost resources: 2 GiB / 1 CPU core; 20 GiB; Debian 10 (Buster)
+   * AWS instance type: t3.small (2 vCPU, 2 GiB); use image ID for Debian Buster, ami-966427e7
+
+ On AWS, an already-existing security group called OnionPerf should be used: this allows ports
+22, 80, 443, 8080, 8443 to the VM. All these ports are necessary for 
+OnionPerf operation.
+
+By default, Greenhost has `iptables` rules in place to only allow ports 22, 80
+and 443 inbound. Details on how to change them to be equivalent to the AWS
+security group rules are given in the following section. 
+
+Finally, DNS for new OnionPerf instances (`*.onionperf.torproject.org`) is
+provided by Tor, and is a manual process: prior to setting up
+the instances, someone from the Admin team (e.g. Hiro) needs to be asked to
+add DNS entries for the hosts.  In the future we might move DNS to AWS for
+automation purposes as part of an Ansible workflow.
+
+### Iptables setup
+
+Greenhost iptables is setup to allow ports 8080 (used by TGen) and 8443 (used by the webserver), in addition to already-enabled ports 22 (SSH for administration purposes), 80 (used by LetsEncrypt) and 443 (used by TGen). Incoming traffic on port 443 is forwarded to port 8080.
+
+```
+sudo iptables -A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
+sudo iptables -A INPUT -p tcp -m tcp --dport 8443 -j ACCEPT
+sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8080
+sudo iptables-save | sudo tee /etc/iptables/rules.v4
+```
+
+AWS iptables is only setup to forward incoming traffic on port 443 to port 8080:
+
+```
+sudo iptables -t nat -A PREROUTING -i eth0 -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8080
+sudo iptables-save | sudo tee /etc/iptables/rules.v4
+```
+
+### Hostname, utilities, and user setup
+
+These rules apply for both AWS and Greenhost VMs with minor exceptions.
+
+```
+sudo apt update && apt upgrade 
+apt install vim # Greenhost only
+```
+
+* Ensure `/etc/hosts` contains `127.0.0.1	localhost {op-hostname}`
+
+```
+sudo echo {op-hostname} | sudo tee /etc/hostname
+```
+
+* The user to administer and run OnionPerf is called `cloud`.
+
+```
+sudo adduser cloud
+sudo adduser cloud sudo 
+sudo mkdir /home/cloud/.ssh
+sudo vim /home/cloud/.ssh/authorized_keys # Add public keys to this file
+sudo chmod -R 600 /home/cloud/.ssh/ 
+```
+
+## Compile Tor from source
+
+The Tor binary will be symlinked into the virtual environment used by
+OnionPerf. Compiling the latest version from source is preferred to installing
+an older version via system packages.
+
+```
+sudo apt install -y automake build-essential libevent-dev libssl-dev zlib1g-dev git
+cd ~/
+git clone https://git.torproject.org/tor.git
+cd tor/
+git checkout -b tor-0.3.5.11 tor-0.3.5.11
+./autogen.sh
+./configure --disable-asciidoc
+make
+src/app/tor --version
+```
+
+## Compile TGen from source
+
+The TGen binary will be symlinked into the virtual environment used by OnionPerf. TGen is not currently packaged in Debian and this is the only method of installation.
+
+```
+sudo apt install -y cmake libglib2.0-dev libigraph0-dev make
+cd ~/
+git clone https://github.com/shadow/tgen.git
+cd tgen/
+mkdir build
+cd build/
+cmake ..
+make
+```
+
+## Install OnionPerf
+
+OnionPerf will be set up as a `systemd` service which persists on reboot. Its output can be viewed by running `systemctl status onionperf`.
+   
+OnionPerf is installed and run in a Python virtual environment. Once the
+software matures, it could be packaged as part of the Debian distribution,
+and run using system packages only.
+To install OnionPerf in a Python virtual environment:
+
+* Create the Python virtual environment and link the previously compiled Tor and TGen binaries.
+
+```
+sudo apt -y install python3-venv
+cd ~/
+python3 -m venv venv
+source venv/bin/activate
+which python3
+ln -s ~/tor/src/app/tor venv/bin/tor
+ln -s ~/tgen/build/src/tgen venv/bin/tgen
+```
+
+* Install OnionPerf and verify installation
+
+```
+git clone https://git.torproject.org/onionperf.git
+pip3 install --no-cache -r onionperf/requirements.txt
+cd onionperf/
+python3 setup.py install
+cd ~/
+onionperf --help
+```
+
+* Create `systemctl` file for OnionPerf
+
+```
+sudo vi /etc/systemd/system/onionperf.service
+```
+
+```
+[Unit]
+Description=OnionPerf
+After=network.target
+
+[Service]
+ExecStart=/bin/bash -c 'cd /home/cloud/ && source venv/bin/activate &&  onionperf measure --tgen-connect-port 443'
+User=cloud
+WorkingDirectory=/home/cloud/
+
+[Install]
+WantedBy=multi-user.target
+```
+
+* Reload the `systemd` changes, start and enable the `onionperf` service
+
+```
+sudo systemctl daemon-reload
+sudo systemctl start onionperf
+```
+
+* Check OnionPerf status
+
+```
+sudo systemctl status onionperf
+```
+
+## LetsEncrypt/Apache Installation
+
+Apache2 is used to serve the OnionPerf directories. It is already setup to persist on reboot, and log files can be found in `/var/log/apache2/`.
+Certbot is automatically setup to renew instance certificates, and logs can be found in `/var/log/letsencrypt`.
+
+To set up and configure Apache with LetsEncrypt certificates:
+
+* Install Apache and Certbot 
+
+```
+sudo apt install -y apache2 certbot python-certbot-apache
+```
+
+* Replace `/etc/apache2/sites-enabled/000-default.conf` with:
+
+```
+<VirtualHost *:80>
+  ServerName op-nl3.onionperf.torproject.net
+  DocumentRoot /home/cloud/onionperf-data
+  <Directory /home/cloud/onionperf-data>
+    SetOutputFilter DEFLATE
+    Options Indexes
+    Require all granted
+  </Directory>
+</VirtualHost>
+```
+
+* Restart Apache to enable the new configuration
+
+```
+sudo systemctl restart apache2
+```
+
+* Obtain Certbot certificate
+
+```
+sudo certbot --apache # choose 2, redirect
+```
+
+* Change port '443' to port '8443' in `/etc/apache2/sites-enabled/000-default-le-ssl.conf`, as well as `/etc/apache2/ports.conf` to move Apache to port 8443.
+Restart Apache to enable the new configuration.
+
+```
+sudo systemctl restart apache2
+```
+
+* The `apache2` user is called `www-data`. To serve data from the OnionPerf
+ directories, owned by the `cloud` user/group, user `www-data` must be added
+ to the `cloud group`, and the directory permissions must be changed
+ to allow group reading.
+
+```
+sudo adduser www-data cloud
+chmod -R 755 ~/onionperf-data/tor-client/
+chmod -R 755 ~/onionperf-data/tor-server/
+```
+
+### Add new htdocs/ URLs to CollecTor
+
+The newly-setup instances must be declared in CollecTor, as follows:
+
+* SSH to the CollecTor VM 
+
+```
+ssh colchicifolium.torproject.org
+```
+
+* Update OnionPerfHosts line in `/srv/collector.torproject.org/collector/collector.properties`
+
+```
+OnionPerfHosts = https://op-us5.onionperf.torproject.net:8443/htdocs/, https://op-hk5.onionperf.torproject.net:8443/htdocs/, https://op-nl5.onionperf.torproject.net:8443/htdocs/, https://op-us4.onionperf.torproject.net:8443/htdocs/, https://op-hk4.onionperf.torproject.net:8443/htdocs/, https://op-nl4.onionperf.torp
+sudo systemctl restart apache2
+```
+
+* Stop and restart CollecTor
+
+```
+cd /srv/collector.torproject.org/collector/
+./collector-service stop
+./collector-service start
+```
+
+## Monit Installation
+
+Monit can only be installed from backports in Debian Buster - this may change
+in the future with new Debian releases.  It runs as a `systemd` service, and
+expects one or more configuration files in `/etc/monit`. The logs generated by
+Monit are available at `/var/log/monit.log`. The emails generated by monit are
+sent to the `metrics-team` mailing list.
+
+
+### Greenhost initial instructions:
+
+* Add `deb http://ftp.{nl,us}.debian.org/debian buster-backports main` to `/etc/apt/sources`, followed by running `sudo apt update`.
+
+### Common Instructions for Greenhost/AWS
+
+* Install Monit
+
+```
+apt install monit
+```
+
+* Change instance names and disks if necessary in `monitrc`, then copy file to `/etc` directory
+
+```
+sudo cp .monitrc /etc/monit/monitrc
+```
+
+* Finally, start and enable monitoring. Enabling the service will warn that `monit.service is not a native service` - this can be ignored, as the service will still start on reboot
+
+```
+sudo systemctl start monit
+sudo systemctl enable monit 
+```
\ No newline at end of file