Initial import

There should only be one running instance of this application. Please do not run it if you are not a Tor developer.

This version of Tor Weather runs on Django version 1.2. It is incompatible with

earlier versions of Django. If you are running Django v1.1, please download the

v1.1-compatible version of this package.

--------------------------------------------------------------------------------

INSTALLATION:

See /doc/INSTALL for details.

------------------------------------------------------------------------------

API:

To generate the Tor Weather API, install epydoc (http://epydoc.sourceforge.net/installing.html) and run the following command within weather: 

	$ epydoc . --config config/epydoc_config.py

Ignore the warnings; they should all pertain to Django-related documentation.

The HTML-formatted API will be in a directory named api/ in weather.

------------------------------------------------------------------------------

DESIGN DOCUMENTATION:

To generate HTML-formatted design documentation, run the following command from 

within doc/:

	$ rst2html --title="Tor Weather Design Documentation" design.txt design.html

To view the documentation, open doc/design.html. If you'd like to view the plaintext design doc instead, see doc/design.txt

-------------------------------------------------------------------------------

TESTING: (For developers)

Run the following command from within weather/:

	$ python manage.py test weatherapp

-------------------------------------------------------------------------------

1) In weather, make a directory named "log".

2) In weather/settings.py, change the EMAIL_* fields to reflect your SMTP 

configurations and the email address for sending the Tor Weather notifications. 

3) In weather.config.config.py, set control_port to the ControlPort specified

by your local torrc if necessary. The default is 9051. Change the base_url field to reflect the base URL for the Tor Weather application.

4) A copy of (or link to) the TorCtl package must be located within the weather 

directory.

5) Create a file named auth_token inside the weather.config directory 

and insert your Tor hashed control password in plain text. The file should 

otherwise be empty.

6) Create the database by running the following command from within the weather

directory:

	$ python manage.py syncdb

7) Look here for documentation concerning how to deploy the Django web 

application:

http://docs.djangoproject.com/en/dev/howto/deployment/

We recommend deploying Django with Apache and mod_wsgi. 

7a) How to deploy weather using Apache and mod_wsgi:

    - $ cat > /home/weather/opt/tor-weather//weather/weather.wsgi << EOF

        import os

        import sys

        sys.path.append('/home/weather/opt/tor-weather')

        sys.path.append('/home/weather/opt/tor-weather/weather')

        sys.path.append('/home/weather/opt/tor-weather/weather/weatherapp')

        os.environ['DJANGO_SETTINGS_MODULE'] = 'settings'

        import django.core.handlers.wsgi

        application = django.core.handlers.wsgi.WSGIHandler()

        EOF

     - Add the following line to your Apache configuration:

       WSGIScriptAlias / /home/weather/opt/tor-weather/weather/weather.wsgi

     - Reload the Apache configuration

8) Once the application is deployed, the listener application should be run from

inside the weather directory:

	$ python manage.py runlistener

The listener waits for consensus events from your local TorCtl process, then 

updates the database and sends notifications.

WARNING: There should only be one instance of this application running at any

    one time. The application does send a single email to new, stable relay

    operators regardless of whether they've subscribed to Tor Weather. We hope

    to strictly limit the number of emails that are sent, so please do not run 

    this application if it is already deployed somewhere.

Copyright (c) 2008, The Tor Project, Inc.

Redistribution and use in source and binary forms, with or without

modification, are permitted provided that the following conditions are

met:

    * Redistributions of source code must retain the above copyright

notice, this list of conditions and the following disclaimer.

    * Redistributions in binary form must reproduce the above

copyright notice, this list of conditions and the following disclaimer

in the documentation and/or other materials provided with the

distribution.

    * Neither the names of the copyright owners nor the names of its

contributors may be used to endorse or promote products derived from

this software without specific prior written permission.

THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS

"AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT

LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR

A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT

OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,

SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT

LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,

DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY

THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT

(INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE

OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.

Tor Weather Design Documentation

================================

.. This file is written in reStructuredText. See the README for instructions on how to generate an HTML version.

Django

------------

Tor Weather was ported to `Django (v1.2)`_ to take advantage of the framework's object relational mapper for database access. Data can be directly accessed by way of Django's models, each of which maps to a table in the database and can be utilized as an object. Django's adaptation of the model-view-controller schema is a bit confusing: controllers are stored in a module named views.py and views are called templates (but models are still, thankfully, called models). Still, Django greatly simplified design and development of the Tor Weather web application, which is why it was chosen.

Directory Structure

-------------------------

Directory structure in Django involves an outer directory, weather, with four 

default files: __init__.py (empty), manage.py (we leave this alone--it syncs the database and runs the server), settings.py (a config file), and urls.py (stores url patterns to recognize and the corresponding controller to direct to). The next level is the application's folder, weatherapp. By default, application folders in Django contain four default files: __init__.py (empty), models.py (a module for the models), tests.py (a module for application tests), and views.py (a module for the controllers).

Within the main directory (weather/), we included a directory for docs (including a README and licensing information), a templates directory including all of the Tor Weather views, and a config directory. In the templates directory, the file base.html is the main template with the Tor banner, and all other template files 'extend' this template (a Django funtionality). The config directory contains a module (templates.py) for mapping template filenames to variables and a module (url_helper.py) for accessing url extensions. Should the template file names or url path extensions in urls.py ever be changed, these need only be updated in the config directory.

---------------------------------------------------------------------------

Tor Weather Notification Types

----------------------------------

The original Tor Weather implementation handled only node-down notifications. You could sign up to receive notifications if a node went down by providing its fingerprint, your email, and a downtime grace period. This implementation of Tor Weather adds three new notification types: T-Shirt notifications, low bandwidth notifications, and version notifications. The user must sign up

for these notifications in order to receive the corresponding emails; by 

default, they will only receive the standard node down notifications.

Each subscription is checked when a consensus document is received on the server running Tor Weather, which are received hourly (although not necessarily at the same time every hour).

Node Down Notifications

.........................

    This is the default notification type. A router is considered "down" if it

    does not appear in the consensus documents *and* does not have a hibernating

    flag set to True in its descriptor file. An email is sent if a node has 

    been down for at least as long as the user's specified grace period and if 

    the user has not yet received a node-down email since the node was last 

    seen up.

T-Shirt Notifications

.......................

    According to the `Tor T-shirt page`_, running a relay can earn the 

    operator a T-shirt in two cases: the router must either allow exits to port 

    80 and average 100 KB/s of traffic for two months of uptime, or the router 

    must average 500 KB/s traffic for two months of uptime (no exit 

    requirement). If the user signs up for T-Shirt notifications, a 

    notification is sent if their router has been seen up for at least 1464 

    hours (61 days, or roughly 2 months) with an average bandwidth of 500 KB/s 

    or 100 KB/s if the router allows exits to port 80. The observed bandwidth 

    is listed in the descriptor file. According to the `directory 

    specifications`_, the listed observed bandwidth "is an estimate of the 

    capacity this server can handle. The server remembers the max bandwidth 

    sustained output over any ten second period in the past day, and another 

    sustained input. The 'observed' value is the lesser of these two numbers." 

    When a new consensus document is received, this observed bandwidth is used 

    to recalculate the average bandwidth for the router being watched. 

Low Bandwidth Notifications

.............................

    We added a notification type to alert users if their router crosses some

    threshold for minimum bandwidth. By default, this threshold is set to 

    20 KB/s, which Tor defines as the minimum bandwidth for a router. Our 

    implementation uses the observed bandwidth listed in the descriptor file

    (see description under T-Shirt Notifications). By this implementation, the 

    notification signals that the router hasn't been able to sustain the 

    minimum threshold for any 10-second period in the past 24 hours.

    *For development: Fine-tune low bandwidth notifications and implement

    a high bandwidth notification to warn users if traffic through their router

    exceeds some ceiling they indicate.*

Version Notifications

........................

    A list of "recommended" Tor versions can be obtained via communication with

    TorCtl. The list contains all stable versions of Tor that are not considered

    obsolete and includes some alpha versions. There are two version 

    notification types to choose from: required and recommended.

        -Required: If the router's version does not appear in the list of

        recommended versions, the user will receive an email notification. 

        -Recommended: If the router's version is not the most recent stable

        (non-alpha/beta) version in the list, the user will receive an email

        notification.

Welcome Email

...................

    The purpose of the welcome email is twofold: first, it advertises Tor 

    Weather as a useful application to new relay operators. Second, it welcomes 

    the relay operator to the Tor community and expresses that their 

    contribution is both significant and appreciated. 

    The database backing Tor Weather stores information about every router seen

    in consensus within the past year (or for as long as Tor Weather has been

    running, whichever is shorter). If a router in the database is flagged as 

    stable, its operator has not been sent a welcome email, and its operator 

    is *not* subscribed to Tor Weather, the router operator's email is parsed 

    from the contact field in the descriptor file for that node. A welcome 

    email containing information about Tor Weather is sent to these node 

    operators if an email address was successfully parsed. 

    The welcome email thanks the operator for his/her contribution and 

    encourages the operator to subscribe to Tor Weather. If the router allows

    exits to port 80, information is appended to the email containing links

    to legal help for exit relay operators. 

    The welcome emails are *not* a subscribable notification type and are sent

    to *all* new, stable relay operators who a) haven't subscribed to Tor 

    Weather and b) provide a parsable (by our standards) email address in the 

    contact field of their configuration file. 

    The welcome email is intended for *new* stable relay operators. To avoid 

    sending the welcome email to long-term relay operators at startup, a 

    48-hour delay period has been implemented immediately following deployment. 

    Any relays added to the database within the first 48-hours following 

    deployment are exempt from the welcome email. That way, the operators who 

    are emailed should largely be new to the network, and relay operators who 

    have been running for a while shouldn't get the welcome email. 

------------------------------------------------------------------------

Modules in weatherapp

--------------------------

models.py

..........

Tor Weather uses three main models: Router, Subscriber, and Subscription. 

To allow flexibility in terms of adding new notification types to Tor Weather, 

each notification type contains its own subclass that inherits from 

Subscription. That way, if a new notification type is added to Tor Weather, it 

won't cause problems with the existing database.

Router

,,,,,,,

    The Router table stores information about every router in the Tor network 

    that a)posts its fingerprint to the directory authorities and b)has been 

    seen up either within the past year or since the new Tor Weather has been 

    running (whichever is more recent). That way, Tor Weather can send welcome 

    notifications to all new relay operators once their node is flagged 

    stable, assuming the email parser successfully gleaned their email address.

Subscriber

,,,,,,,,,,,,

    The Subscriber model stores information about the user, specifically their

    email address, authorization keys (used to access user-specific pages for

    confirming their subscription, changing preferences, and unsubscribing), 

    and the router to which they are subscribed (represented by a many-to-one 

    foreign key relationship to the corresponding Router object).

Subscription

,,,,,,,,,,,,,

    We decided to create separate Subscription models to improve flexibility: 

    with new "subscription types", each Subscriber can have multiple 

    Subscriptions to a Router. The subscription types (each of which inherits 

    from the Subscription model) include NodeDownSub (sends an email if the 

    Router is down), TShirtSub (sends an email if the Router earned the 

    Subscriber--ideally the operator--a t-shirt), LowBandwidthSub (sends an 

    email if the Router's bandwidth drops below 50 KB/s, Tor's minimum for 

    being a functional node), and VersionSub (sends an email if the Router is 

    running an out-of-date version of Tor).

views.py

............

The module stores all of the controllers for the web interface.

auth_token

..............

A key used to authenticate a connection to the local Tor process through TorCtl

ctlutil.py

............

Contains the CtlUtil class, which establishes a connection to TorCtl and handles all relevant communication (i.e. accessing consensus documents and descriptor files).

emails.py

............

Contains all email messages that Tor Weather can send and the methods to send them.

error_messages.py

.................

Contains the error messages that are passed to the error template and the

method to access the appropriate message.

listener.py

..............

Listens to TorCtl for new consensus events and initializes the events to check all subscriptions.

updaters.py

.............

Calls methods for checking each Subscription in the database and populating/updating the Router table.

-------------------------------------------------------------------------

Event Flow

------------------

The arrival of a new consensus document triggers a cascade of events. First, the Router table is populated and updated, and welcome emails are sent to new stable relay operators. Next, each Subscription is checked to determine if a Tor Weather notification should be sent. Finally, all of the necessary emails are sent out.

Consensus Event Hanlding

..........................

The main method in listeners.py spawns a thread that continually listens for new consensus events through TorCtl and triggers run_all() within the updaters module when a consensus document is received.

Updating and Notifying

..........................

Updating the database and sending email notifications is handled by the updaters module. Within updaters, the Router table is first populated and updated by scanning the consensus document for all router fingerprints therein. If a router isn't stored in the database, it's added. Relevant information for existing routers in the database is updated using descriptor information. If a router in the database is flagged as stable in the consensus document, a welcome email is sent to the node operator after parsing their email from the descriptor file. The welcome email contains information about Tor Weather and legal information if the node functions as an exit node.

Each Subscription in the database is updated, and emails are sent to the Subscriber's email if the conditions indicate a notification should be sent. 

.. _Django (v1.2): http://docs.djangoproject.com/en/1.2/intro/overview/

.. _Tor T-Shirt page: http://www.torproject.org/tshirt.html.en

.. _directory specifications: https://svn.torproject.org/svn/tor/trunk/doc/spec/dir-spec.txt