Skip to content
Snippets Groups Projects

setup

Last edited by Hiro

Instructions for setting up long-term instances

Environment setup

For running OnionPerf instances, we consider AWS or Greenhost VMs, with the following setup:

  • Greenhost resources: 2 GiB / 1 CPU core; 20 GiB; Debian 10 (Buster)
  • AWS instance type: t3.small (2 vCPU, 2 GiB); use image ID for Debian Buster, ami-966427e7

On AWS, an already-existing security group called OnionPerf should be used: this allows ports 22, 80, 443, 8080, 8443 to the VM. All these ports are necessary for OnionPerf operation.

By default, Greenhost has iptables rules in place to only allow ports 22, 80 and 443 inbound. Details on how to change them to be equivalent to the AWS security group rules are given in the following section.

Finally, DNS for new OnionPerf instances (*.onionperf.torproject.org) is provided by Tor, and is a manual process: prior to setting up the instances, someone from the Admin team needs to be asked to add DNS entries for the hosts. This can be done by opening a ticket under https://gitlab.torproject.org/tpo/tpa/team/-/issues. In the future we might move DNS to AWS for automation purposes as part of an Ansible workflow.

Iptables setup

Greenhost iptables is setup to allow ports 8080 (used by TGen) and 8443 (used by the webserver), in addition to already-enabled ports 22 (SSH for administration purposes), 80 (used by LetsEncrypt) and 443 (used by TGen). Incoming traffic on port 443 is forwarded to port 8080. To set the port forwarding rule on the local interface, the interface name (e.g. eth0) must first be identified with ip link show.

sudo iptables -A INPUT -p tcp -m tcp --dport 8080 -j ACCEPT
sudo iptables -A INPUT -p tcp -m tcp --dport 8443 -j ACCEPT
sudo iptables -t nat -A PREROUTING -i {interface_name} -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8080
sudo iptables-save | sudo tee /etc/iptables/rules.v4

AWS iptables is only setup to forward incoming traffic on port 443 to port 8080. An iptables directory must be created before saving the iptables rules:

sudo iptables -t nat -A PREROUTING -i {interface_name} -p tcp -m tcp --dport 443 -j REDIRECT --to-ports 8080
sudo mkdir /etc/iptables
sudo iptables-save | sudo tee /etc/iptables/rules.v4

To check the status of the port forwarding rule:

sudo iptables -t nat -L

If you plan to monitor the instance via prometheus you have to allow the VM to open the port 9100 for the node exporter to be reached.

If you plan to run an adhoc monitoring exporter you will also remember to open the corresponding ports.

Hostname, utilities, and user setup

These rules apply for both AWS and Greenhost VMs with minor exceptions.

sudo apt update && apt upgrade
apt install vim # Greenhost only
  • Ensure /etc/hosts contains 127.0.0.1 localhost {op-hostname} 
sudo echo {op-hostname} | sudo tee /etc/hostname
  • The user to administer and run OnionPerf is called cloud.
sudo adduser cloud
sudo adduser cloud sudo
sudo mkdir /home/cloud/.ssh
sudo vim /home/cloud/.ssh/authorized_keys # Add public keys to this file
sudo chmod -R 600 /home/cloud/.ssh/

Compile Tor from source

The Tor binary will be symlinked into the virtual environment used by OnionPerf. Compiling the latest version from source is preferred to installing an older version via system packages.

sudo apt install -y automake build-essential libevent-dev libssl-dev zlib1g-dev git
cd ~/
git clone https://git.torproject.org/tor.git
cd tor/
git checkout -b tor-0.3.5.11 tor-0.3.5.11
./autogen.sh
./configure --disable-asciidoc
make
src/app/tor --version

Compile TGen from source

The TGen binary will be symlinked into the virtual environment used by OnionPerf. TGen is not currently packaged in Debian and this is the only method of installation.

sudo apt install -y cmake libglib2.0-dev libigraph0-dev make
cd ~/
git clone https://github.com/shadow/tgen.git
cd tgen/
mkdir build
cd build/
cmake ..
make

Install OnionPerf

OnionPerf will be set up as a systemd service which persists on reboot. Its output can be viewed by running systemctl status onionperf.

OnionPerf is installed and run in a Python virtual environment. Once the software matures, it could be packaged as part of the Debian distribution, and run using system packages only. To install OnionPerf in a Python virtual environment:

  • Create the Python virtual environment and link the previously compiled Tor and TGen binaries.
sudo apt -y install python3-venv
cd ~/
python3 -m venv venv
source venv/bin/activate
which python3
ln -s ~/tor/src/app/tor venv/bin/tor
ln -s ~/tgen/build/src/tgen venv/bin/tgen
  • Install OnionPerf and verify installation
git clone https://git.torproject.org/onionperf.git
pip3 install --no-cache -r onionperf/requirements.txt
cd onionperf/
python3 setup.py install
cd ~/
onionperf --help
  • Create systemctl file for OnionPerf
sudo vi /etc/systemd/system/onionperf.service
[Unit]
Description=OnionPerf
After=network.target

[Service]
ExecStart=/bin/bash -c 'cd /home/cloud/ && source venv/bin/activate &&  onionperf measure --tgen-connect-port 443'
User=cloud
WorkingDirectory=/home/cloud/

[Install]
WantedBy=multi-user.target
  • Reload the systemd changes, start and enable the onionperf service
sudo systemctl daemon-reload
sudo systemctl start onionperf
  • Check OnionPerf status
sudo systemctl status onionperf

LetsEncrypt/Apache Installation

Apache2 is used to serve the OnionPerf directories. It is already setup to persist on reboot, and log files can be found in /var/log/apache2/. Certbot is automatically setup to renew instance certificates, and logs can be found in /var/log/letsencrypt.

To set up and configure Apache with LetsEncrypt certificates:

  • Install Apache and Certbot
sudo apt install -y apache2 certbot python-certbot-apache
  • Replace /etc/apache2/sites-enabled/000-default.conf with:
<VirtualHost *:80>
  ServerName op-nl3.onionperf.torproject.net
  DocumentRoot /home/cloud/onionperf-data
  <Directory /home/cloud/onionperf-data>
    SetOutputFilter DEFLATE
    Options Indexes
    Require all granted
  </Directory>
</VirtualHost>
  • Restart Apache to enable the new configuration
sudo systemctl restart apache2
  • Obtain Certbot certificate
sudo certbot --apache # choose 2, redirect
  • Change port '443' to port '8443' in /etc/apache2/sites-enabled/000-default-le-ssl.conf, as well as /etc/apache2/ports.conf to move Apache to port 8443. Restart Apache to enable the new configuration.
sudo systemctl restart apache2
  • The apache2 user is called www-data. To serve data from the OnionPerf directories, owned by the cloud user/group, user www-data must be added to the cloud group, and the directory permissions must be changed to allow group reading.
sudo adduser www-data cloud
chmod -R 755 ~/onionperf-data/tor-client/
chmod -R 755 ~/onionperf-data/tor-server/

Add new htdocs/ URLs to CollecTor

The newly-setup instances must be declared in CollecTor, as follows:

  • SSH to the CollecTor VM
ssh colchicifolium.torproject.org
  • Sudo as the collector user
sudo -u collector -s
  • Update OnionPerfHosts line in /srv/collector.torproject.org/collector/collector.properties 
OnionPerfHosts = https://op-us5.onionperf.torproject.net:8443/htdocs/, https://op-hk5.onionperf.torproject.net:8443/htdocs/, https://op-nl5.onionperf.torproject.net:8443/htdocs/, https://op-us4.onionperf.torproject.net:8443/htdocs/, https://op-hk4.onionperf.torproject.net:8443/htdocs/, https://op-nl4.onionperf.torproject.net:8443
  • Stop and restart CollecTor
cd /srv/collector.torproject.org/collector/
./collector-service stop
./collector-service start

Monit Installation

Monit can only be installed from backports in Debian Buster - this may change in the future with new Debian releases. It runs as a systemd service, and expects one or more configuration files in /etc/monit. The logs generated by Monit are available at /var/log/monit.log. The emails generated by monit are sent to the metrics-alerts mailing list.

Greenhost initial instructions:

  • Add deb http://ftp.{nl,us}.debian.org/debian buster-backports main to /etc/apt/sources, followed by running sudo apt update.

Common Instructions for Greenhost/AWS

  • Install Prometheus
apt install prometheus-node-exporter

Environment setup

Don't forget to request the removal of all old/unused Onionperf DNS entries, by opening a ticket under https://gitlab.torproject.org/tpo/tpa/team/-/issues.

Comments

    Please register or sign in to add a comment.