Table of Contents
- 1. TODO Name
- 2. TODO Synopsis
- 3. DONE Contacts
- 4. TODO Overview
- 5. DONE Sources
- 6. TODO Deployment
- 7. TODO Diagnostics
- 8. TODO Monitoring
[0/2] - 9. DONE Disaster Recovery
- 10. TODO Service Level Agreement
- 11. TODO See Also
- 12. TODO Standards
- 13. TODO History
- 14. TODO Authors
- 15. DONE Major Caveats
- 16. DONE Bugs
1 TODO Name
exit-ops - Exit Scanner, TorDNSEL and Tor Check Operations
2 TODO Synopsis
While the three services described in this document could be implemented as discrete components, they currently have tight coupling which means they must all be deployed on the same host.
2.1 TODO Exit Scanner [0/3]
The exit scanner performs active measurement of Tor exit relays in order to determine the IP addresses that are used for exit connections. The active measurement uses an exitmap module, which is wrapped in a script to produce output formatted as an [Exit List](https://metrics.torproject.org/collector.html#type-tordnsel).
The exit list results are consumed by CollecTor, [TorDNSEL](tordnsel) and [Tor Check](../check-ops/). Exit lists and bulk exit lists are also consumed by third-party external applications at the following URLs:
- https://check.torproject.org/exit-addresses - Latest exit list
- https://check.torproject.org/torbulkexitlist - Latest bulk exit list
Documentation questions:
[ ]How long do we keep old measurements in the exit list?[ ]What are the timings for measurement runs?[ ]How many old exit lists do we keep around?
2.2 TODO TorDNSEL [0/2]
TorDNSEL is a DNS list service that behaves in a similar way to Domain Name System-based Blackhole Lists. IP addresses will give positive results in the event that an address has been found to be used by an exit relay in a recent scan.
Documentation questions:
[ ]For how long does an address give a positive result?[ ]Do we also include all IP addresses of exit flagged relays in the consensus?
2.3 TODO Tor Check [0/1]
Tor Check is a website that can be used to determine if a browser is using the Tor network for queries. It will also check the User-Agent to determine if a user is using Tor Browser. It is accessed via HTTPS at https://check.torproject.org/.
Documentation questions:
[ ]Where is the JSON API?
3 DONE Contacts
The primary contact for this service is the Metrics Team <metrics-team@lists.torproject.org>. For urgent queries, contact karsten, irl, or gaba in .
4 TODO Overview
The underlying infrastructure for the exit scanner, TorDNSEL and Tor Check services is provided by the
Tor Sysadmin Team (TSA). All services run on one virtual machine with the hostname check-01.torproject.org.
4.1 TODO Exit Scanner
Documentation questions:
[ ]Where is the exitmap module?[ ]What are the services called?[ ]What user is used?
4.2 TODO TorDNSEL
Documentation questions:
[ ]Where does the zone file live?[ ]Ticket about doing DNSSEC signing[ ]Where is DNS served[ ]What name is delegated[ ]Can delegation work in testing environment?
5 DONE Sources
The sources for exitmap are available on GitHub: https://github.com/NullHypothesis/exitmap. The exitmap wrapper and module used by the exit scanner can be found in the metrics-cloud repository.
The wrapper script is also responsible for writing out the zone file to be used by the TorDNSEL service and triggering a reload of the zone.
The sources for Tor Check are available in our git: https://gitweb.torproject.org/check.git.
6 TODO Deployment
6.1 DONE Initial deployment
The initial deployment procedure is split into 2 parts:
- System setup
- Installing and starting the services
There are no manual steps required to load state, and backups do not need to be performed for the host running this service. Everything can be configured from scratch with only the Ansible playbook.
6.1.1 DONE Development/testing in AWS
For development or testing in AWS, a CloudFormation template is available named exit-scanner-dev.yml.
From the CloudFormation portal, select your stack and view the outputs. You will find here the public IP address for the EC2 instance that has been created. Add this instance to ansible/dev in your local copy of metrics-cloud.git under "[exit-scanners]".
You can now setup the machine with Ansible by running:
``` ansible-playbook -i dev exit-scanners-aws.yml ```
Note that the AWS AMI used has passwordless sudo, so no password need be given.
6.1.2 DONE Fresh machine from TSA
Add the host name of the new instance to ansible/production in your local copy of metrics-cloud.git under "[exit-scanners]" and commit the change.
You can now setup the machine with Ansible by running:
``` ansible-playbook -i production -K exit-scanners.yml ```
6.2 TODO Upgrade [0/2]
The upstream sources for the applications that make up this service do not have managed releases which makes this difficult.
To fix a bug in the exit scanner wrapper script, fix this in the metrics-cloud repository and re-run the deployment playbook.
[ ]Can we upgrade exitmap sensibly?[ ]Can we upgrade Tor Check sensibly?
7 TODO Diagnostics
7.1 TODO Logs [0/2]
[ ]What things log?[ ]Where do the logs go?
8 TODO Monitoring [0/2]
[ ]CollecTor log messages[ ]Nagios
9 DONE Disaster Recovery
The exit scanner service does not need to maintain any state between runs. It's nice if it can in order to cope with a relay that happened to be down at the time we tried to measure it but in the event of a failure it is perfectly acceptable to throw away the old box and provision a new one. Follow the initial deployment instructions above.
10 TODO Service Level Agreement
11 TODO See Also
12 TODO Standards
The exit scanner service produces exit lists according to the TorDNSEL exit list format.
13 TODO History
14 TODO Authors
15 DONE Major Caveats
The exit scanner service does not support IPv6.
16 DONE Bugs
Known bugs can be found in the Tor Project Trac for:
For bugs relating to exitmap, they are found on the GitHub project: https://github.com/NullHypothesis/exitmap/issues
New bug reports should be filed in the appropriate tracker and component.