Commit 454c0450 authored by Karsten Loesing's avatar Karsten Loesing
Browse files

Merge branch 'task-40001' into develop

parents 83e4b525 e3051287
This diff is collapsed.
# Minimal makefile for Sphinx documentation
#
# You can set these variables from the command line.
SPHINXOPTS =
SPHINXBUILD = sphinx-build
SPHINXPROJ = onionperf
SOURCEDIR = .
BUILDDIR = _build
# Put it first so that "make" without argument is like "make help".
help:
@$(SPHINXBUILD) -M help "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
.PHONY: help Makefile
# Catch-all target: route all unknown targets to Sphinx using the new
# "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS).
%: Makefile
@$(SPHINXBUILD) -M $@ "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)
\ No newline at end of file
# -*- coding: utf-8 -*-
#
# Configuration file for the Sphinx documentation builder.
#
# This file does only contain a selection of the most common options. For a
# full list see the documentation:
# http://www.sphinx-doc.org/en/master/config
# -- Path setup --------------------------------------------------------------
# If extensions (or modules to document with autodoc) are in another directory,
# add these directories to sys.path here. If the directory is relative to the
# documentation root, use os.path.abspath to make it absolute, like shown here.
#
import os
import sys
sys.path.insert(0, os.path.abspath('..'))
# -- Project information -----------------------------------------------------
project = 'onionperf'
copyright = '2019, Ana Custura'
author = 'Ana Custura'
# The short X.Y version
version = ''
# The full version, including alpha/beta/rc tags
release = ''
# -- General configuration ---------------------------------------------------
# If your documentation needs a minimal Sphinx version, state it here.
#
# needs_sphinx = '1.0'
# Add any Sphinx extension module names here, as strings. They can be
# extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
# ones.
extensions = [
'sphinx.ext.autodoc',
'sphinx.ext.intersphinx',
'sphinx.ext.viewcode',
]
# Add any paths that contain templates here, relative to this directory.
templates_path = ['_templates']
# The suffix(es) of source filenames.
# You can specify multiple suffix as a list of string:
#
# source_suffix = ['.rst', '.md']
source_suffix = '.rst'
# The master toctree document.
master_doc = 'index'
# The language for content autogenerated by Sphinx. Refer to documentation
# for a list of supported languages.
#
# This is also used if you do content translation via gettext catalogs.
# Usually you set "language" from the command line for these cases.
language = None
# List of patterns, relative to source directory, that match files and
# directories to ignore when looking for source files.
# This pattern also affects html_static_path and html_extra_path .
exclude_patterns = ['_build', 'Thumbs.db', '.DS_Store']
# The name of the Pygments (syntax highlighting) style to use.
pygments_style = 'sphinx'
# -- Options for HTML output -------------------------------------------------
# The theme to use for HTML and HTML Help pages. See the documentation for
# a list of builtin themes.
#
html_theme = 'alabaster'
# Theme options are theme-specific and customize the look and feel of a theme
# further. For a list of options available for each theme, see the
# documentation.
#
# html_theme_options = {}
# Add any paths that contain custom static files (such as style sheets) here,
# relative to this directory. They are copied after the builtin static files,
# so a file named "default.css" will overwrite the builtin "default.css".
html_static_path = ['_static']
# Custom sidebar templates, must be a dictionary that maps document names
# to template names.
#
# The default sidebars (for documents that don't match any pattern) are
# defined by theme itself. Builtin themes are using these templates by
# default: ``['localtoc.html', 'relations.html', 'sourcelink.html',
# 'searchbox.html']``.
#
# html_sidebars = {}
# -- Options for HTMLHelp output ---------------------------------------------
# Output file base name for HTML help builder.
htmlhelp_basename = 'onionperfdoc'
# -- Options for LaTeX output ------------------------------------------------
latex_elements = {
# The paper size ('letterpaper' or 'a4paper').
#
# 'papersize': 'letterpaper',
# The font size ('10pt', '11pt' or '12pt').
#
# 'pointsize': '10pt',
# Additional stuff for the LaTeX preamble.
#
# 'preamble': '',
# Latex figure (float) alignment
#
# 'figure_align': 'htbp',
}
# Grouping the document tree into LaTeX files. List of tuples
# (source start file, target name, title,
# author, documentclass [howto, manual, or own class]).
latex_documents = [
(master_doc, 'onionperf.tex', 'onionperf Documentation',
'Ana Custura', 'manual'),
]
# -- Options for manual page output ------------------------------------------
# One entry per manual page. List of tuples
# (source start file, name, description, authors, manual section).
man_pages = [
(master_doc, 'onionperf', 'onionperf Documentation',
[author], 1)
]
# -- Options for Texinfo output ----------------------------------------------
# Grouping the document tree into Texinfo files. List of tuples
# (source start file, target name, title, author,
# dir menu entry, description, category)
texinfo_documents = [
(master_doc, 'onionperf', 'onionperf Documentation',
author, 'onionperf', 'One line description of project.',
'Miscellaneous'),
]
# -- Extension configuration -------------------------------------------------
# -- Options for intersphinx extension ---------------------------------------
# Example configuration for intersphinx: refer to the Python standard library.
intersphinx_mapping = {'https://docs.python.org/': None}
.. onionperf documentation master file, created by
sphinx-quickstart on Thu Jan 24 19:39:13 2019.
You can adapt this file completely to your liking, but it should at least
contain the root `toctree` directive.
Welcome to onionperf's documentation!
=====================================
.. toctree::
:maxdepth: 3
:caption: Contents:
onionperf
Indices and tables
==================
* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`
What is OnionPerf?
==================
OnionPerf is a utility to track Tor and Onion service performance.
OnionPerf uses multiple processes and threads to download random data through
Tor while tracking the performance of those downloads. The data is served and
fetched on localhost using two TGen (traffic generator) processes, and is
transferred through Tor using Tor client processes and an ephemeral Tor Onion
service. Tor control information and TGen performance statistics are logged to
disk, analyzed once per day to produce a json stats database and files that can
feed into Torperf, and can later be used to visualize changes in Tor client
performance over time.
Installation
============
OnionPerf depends on traffic generator TGen. Instructions for TGen installation can also be found at https://github.com/shadow/tgen.
To install it:
Tgen
----
1. Install dependencies
::
apt install cmake make build-essential gcc libigraph0-dev libglib2.0-dev
2. Clone repositories
::
git clone https://github.com/shadow/tgen.git
3. Build and install
::
mkdir build
cd build
cmake .. -DCMAKE_INSTALL_PREFIX=/home/$USER/.local
make
ln -s build/tgen /usr/bin/tgen
OnionPerf
---------
Once TGen is installed, follow these instructions to install OnionPerf:
1. Clone repositories
::
apt install git
git clone https://git.torproject.org/onionperf.git
2. Install dependencies
::
apt install tor libxml2-dev python-dev python-lxml python-networkx python-scipy python-matplotlib python-numpy python-netifaces python-ipaddress
Ensure stem is the latest version for v3 Onion services to work, this can be installed from backports:
::
echo 'deb http://deb.debian.org/debian stretch-backports main' >> /etc/apt/sources.list
apt update
apt-get -t stretch-backports install python-stem
3. Install OnionPerf
::
cd onionperf
sudo python setup.py install
4. Run an OnionPerf test measurement
::
onionperf measure --oneshot --onion-only
Watch out for any projects in the log. The output of the measurement should look similar to the following:
::
2019-03-05 17:57:02 1551805022.995195 [onionperf] [INFO] Using 'tor' binary at /usr/bin/tor
2019-03-05 17:57:02 1551805022.995359 [onionperf] [INFO] Using 'tgen' binary at /home/ana/Work/shadow/src/plugin/shadow-plugin-tgen/build/tgen
2019-03-05 17:57:03 1551805023.005096 [onionperf] [INFO] Bootstrapping started...
2019-03-05 17:57:03 1551805023.005265 [onionperf] [INFO] Log files for the client and server processes will be placed in /home/ana/onionperf-data
2019-03-05 17:57:03 1551805023.005340 [onionperf] [INFO] Starting TGen server process on port 8080...
2019-03-05 17:57:03 1551805023.012559 [onionperf] [INFO] TGen server running at 0.0.0.0:8080
2019-03-05 17:57:03 1551805023.012727 [onionperf] [INFO] Logging TGen server process output to /home/ana/onionperf-data/tgen-server/onionperf.tgen.log
2019-03-05 17:57:03 1551805023.013463 [onionperf] [INFO] Starting Tor server process with ControlPort=26984, SocksPort=17674...
2019-03-05 17:57:03 1551805023.016020 [onionperf] [INFO] Logging Tor server process output to /home/ana/onionperf-data/tor-server/onionperf.tor.log
2019-03-05 17:57:18 1551805038.183236 [onionperf] [INFO] Logging Tor server control port monitor output to /home/ana/onionperf-data/tor-server/onionperf.torctl.log
2019-03-05 17:57:53 1551805073.334130 [onionperf] [INFO] Creating ephemeral hidden service...
2019-03-05 17:57:55 1551805075.847095 [onionperf] [INFO] Ephemeral hidden service is available at p3d2xcwjevqkiwtyejjbjxwadp5ces7v4k4hhrsheqwbbokuismkiyad.onion
2019-03-05 17:57:55 1551805075.851187 [onionperf] [INFO] Starting Tor client process with ControlPort=18843, SocksPort=18397...
2019-03-05 17:57:55 1551805075.851770 [onionperf] [INFO] Logging Tor client process output to /home/ana/onionperf-data/tor-client/onionperf.tor.log
2019-03-05 17:58:49 1551805129.399007 [onionperf] [INFO] Logging Tor client control port monitor output to /home/ana/onionperf-data/tor-client/onionperf.torctl.log
2019-03-05 17:58:52 1551805132.401905 [onionperf] [INFO] Starting TGen client process on port 14785...
2019-03-05 17:58:52 1551805132.408057 [onionperf] [INFO] Logging TGen client process output to /home/ana/onionperf-data/tgen-client/onionperf.tgen.log
2019-03-05 17:58:52 1551805132.414970 [onionperf] [INFO] Bootstrapping finished, entering heartbeat loop
2019-03-05 17:58:53 1551805133.416607 [onionperf] [INFO] Onionperf is running in Oneshot mode. It will download a 5M file and shut down gracefully...
2019-03-05 17:59:24 1551805164.697978 [onionperf] [INFO] Onionperf has downloaded a 5M file in oneshot mode, and will now shut down.
2019-03-05 17:59:24 1551805164.698091 [onionperf] [INFO] Cleaning up child processes now...
2019-03-05 17:59:24 1551805164.707111 [onionperf] [INFO] Joining tgen_server_watchdog thread...
2019-03-05 17:59:25 1551805165.094310 [onionperf] [INFO] Joining tor_server_watchdog thread...
2019-03-05 17:59:25 1551805165.094690 [onionperf] [INFO] Joining torctl_server_helper thread...
2019-03-05 17:59:25 1551805165.113840 [onionperf] [INFO] command '/home/ana/Work/shadow/src/plugin/shadow-plugin-tgen/build/tgen /home/ana/onionperf-data/tgen-client/tgen.graphml.xml' finished as expected
2019-03-05 17:59:25 1551805165.493231 [onionperf] [INFO] Joining tor_client_watchdog thread...
2019-03-05 17:59:25 1551805165.493633 [onionperf] [INFO] Joining torctl_client_helper thread...
2019-03-05 17:59:25 1551805165.538060 [onionperf] [INFO] Joining tgen_client_watchdog thread...
2019-03-05 17:59:25 1551805165.538464 [onionperf] [INFO] Joining logrotate thread...
2019-03-05 17:59:26 1551805166.539878 [onionperf] [INFO] Child processes terminated
2019-03-05 17:59:26 1551805166.540305 [onionperf] [INFO] Child process cleanup complete!
2019-03-05 17:59:26 1551805166.540603 [onionperf] [INFO] Exiting
Deployment
==========
Deployment options
------------------
There are
various options available for deployment of measurements. These options can be
passed to OnionPerf via command line arguments, as follows:
::
--onion-only
Only download files through an Onion service (default: disabled) ::
--inet-only
Only download files through the Internet (default: disabled) ::
--single-onion
Use a single Onion service, which uses a direct circuit between the Onion
service and the introduction and rendezvous point. ::
--torclient-conf-file FILE
Download files using specified configuration file for the Tor client (default: disabled) ::
--torserver-conf-file FILE
In addition to specifying configuration files, you can pass newline-separated
Tor configuration options to the Tor process by adding them to the
:code:`BASETORRC` environment variable. These options are prepended to all other
configuration options. Here is an example which prepends the options
:code:`Option1 Foo` and :code:`Option2 Bar` to Tor's configuration file: ::
BASETORRC=$'Option1 Foo\nOption2 Bar' onionperf ...
Download files using specified configuration file for the Tor server (default: disabled) ::
--additional-client-conf STRING
Download files using specified configuration lines (default: disabled)
By default, OnionPerf downloads files using both the Internet and Onion services, using a v3 Onion address.
It uses publicly available relays, but by specifying additional configuration files it can be configured to run
on test Tor networks, or using bridges with or without pluggable transports.
::
--oneshot
Only download a 5M file and then shut down gracefully (default: disabled)
By default, OnionPerf runs continuously and appends measurement information to
log files as they happen. At midnight, the log files are rotated and the measurement continues.
A oneshot measurement will run only until one successful download has completed.
::
--nickname STRING
The 'SOURCE' STRING to use in stats files produced by OnionPerf (default: hostname of the current machine)
::
--prefix PATH
A directory PATH prefix where OnionPerf will run (default: current directory)
::
--tor PATH
A file PATH to a Tor binary (default: looks in $PATH)
::
--tgen PATH
A file PATH to a TGen binary (default: looks in $PATH)
Example vanilla Tor deployment
------------------------------
The following command will download files continuously using a Tor client through Onion service version 3 and via the Internet until it is stopped:
::
onionperf --measure
Example vanilla bridge deployment
---------------------------------
The following command will download files continuously using a Tor client through Onion service version 3 and via the Internet until it is stopped.
The Tor client will always pick one of the bridges provided in this configuration to be the first hop in the circuits it builds:
::
onionperf --measure --additional-client-conf="UseBridges 1
Bridge 72.14.177.231:9001 AC0AD4107545D4AF2A595BC586255DEA70AF119D
Bridge 195.91.239.8:9001 BA83F62551545655BBEBBFF353A45438D73FD45A
Bridge 148.63.111.136:35577 768C8F8313FF9FF8BBC915898343BC8B238F3770"
Note: a new line must be added at the end of each configuration directive.
A second way of passing this configuration to OnionPerf would be to create a file called tor_conf in a directory of your choice, containing the lines:
::
UseBridges 1
Bridge 148.63.111.136:35577 768C8F8313FF9FF8BBC915898343BC8B238F3770
Bridge 195.91.239.8:9001 BA83F62551545655BBEBBFF353A45438D73FD45A
Bridge 148.63.111.136:35577 768C8F8313FF9FF8BBC915898343BC8B238F3770
This file is then passed to the client configurator in OnionPerf:
::
onionperf --measure --torclient-config-file=/path/to/tor_conf
If we want to use vanilla Tor for the client, but download the files through an Onion service accessible via a bridge, the same configuration file containing the bridge lines can be passed to the server:
::
onionperf --measure --torserver-config-file=/path/to/tor_conf
Note that bridge lines for configuration can be downloaded from https://bridges.torproject.org.
Example bridge with Pluggable Transport deployment
--------------------------------------------------
Similarly to the above, the Tor client can use Pluggable Transports (PT) with bridges. Here we present examples for meek and obfs4proxy.
You must have the meek and/or obfs4proxy binaries installed. The binaries can
be obtained by downloading the latest version of Tor browser bundle, or they
can be installed from source. In the example file that follows, directive "ClientTransportPlugin"
needs to point to the path of the binary corresponding to the wanted PT. Finally, both meek and
obfs4 enabled bridges can be obtained from the bridge database.
Example file torrc1:
::
UseBridges 1
# Example meek bridge line - meek bridge lines can be downloaded from https://bridges.torproject.org
Bridge meek 0.0.2.0:1 url=https://at-b2.erg.abdn.ac.uk
# meek configuration
ClientTransportPlugin meek exec /usr/bin/meek
Example file torrc2:
::
# Example obfs4 bridge - meek bridge lines can be downloaded from bridges.torproject.org
Bridge obfs4 137.50.19.19:5001 AE77C35CAC66C2F207319939029D6D22945BDA84 cert=kwpT6sHRa80CnoSCGzelo2wl4RU7cC+mjBCihj2gAJAnvNyTWD3Pk9Ae05+fGpiGzHleOw iat-mode=0
# obfs4 configuration
ClientTransportPlugin obfs2,obfs3,obfs4,scramblesuit exec /usr/bin/obfs4proxy
Then, the configuration files containing the required bridge and PT lines can be passed to the either the Tor server or client:
::
onionperf --measure --torserver-config-file=/path/to/torrc1 --torclient-config-file=/path/to/torrc2
In this example, the Onion services uses the obfs4 bridge configured in file torrc2 to connect to the Tor network, while the client uses the meek bridge configured in file torrc1 to connect to the Tor network.
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment