|
|
# exit-ops
|
|
|
|
|
|
Exit Scanner, TorDNSEL and Tor Check Operations
|
|
|
|
|
|
[[!toc levels=3]]
|
|
|
|
|
|
# Overview
|
|
|
|
|
|
While the three services described in this document could be implemented as discrete components,
|
|
|
they currently have tight coupling which means they must all be deployed on the same host.
|
|
|
|
|
|
```
|
|
|
+------------+
|
|
|
| Exit +---------------------------------------+
|
|
|
| Scanner |-----------------------------+ |
|
|
|
| +----------> DNS Zone File | |
|
|
|
+--------+---+ + | |
|
|
|
launches| ^ | | |
|
|
|
v |gets results v v |
|
|
|
+-------+ +---+--------+ +----+-----+ Bulk |
|
|
|
| Tor +^------+ exitmap | | TSA | Exit |
|
|
|
| Client+------v+ | | Managed | List |
|
|
|
+-------+ +------------+ | BIND | |
|
|
|
using ipscan module +----------+ |
|
|
|
|
|
|
|
v
|
|
|
+--------+ +-----------+ Exit List
|
|
|
| Tor +<---------+ Tor | +
|
|
|
| Client | | Check +<----------+
|
|
|
| +--------->+ |
|
|
|
+--------+ +-----+-----+
|
|
|
^
|
|
|
|
|
|
|
+-----+-----+
|
|
|
| Apache |
|
|
|
| Reverse |
|
|
|
| Proxy |
|
|
|
+-----------+
|
|
|
```
|
|
|
|
|
|
## Exit Scanner
|
|
|
|
|
|
The exit scanner performs active measurement of Tor exit relays in order to determine the IP addresses that are used for exit connections.
|
|
|
The active measurement uses an exitmap module, which is wrapped in a script to produce output formatted as an [Exit List](<https://metrics.torproject.org/collector.html#type-tordnsel>).
|
|
|
|
|
|
The exit list results are consumed by CollecTor, [TorDNSEL](tordnsel) and [Tor Check](../check-ops/).
|
|
|
Exit lists and bulk exit lists are also consumed by third-party external applications at the following URLs:
|
|
|
|
|
|
- <https://check.torproject.org/exit-addresses> - Latest exit list
|
|
|
- <https://check.torproject.org/torbulkexitlist> - Latest bulk exit list
|
|
|
|
|
|
Previous scan results are kept in the latest exit list for up to 48 hours since
|
|
|
they were last seen. Scans will run every 40 minutes, or continuously if scans
|
|
|
take longer than 40 minutes to complete.
|
|
|
|
|
|
There is currently no process that will clean up old exit lists from the exit
|
|
|
list server.
|
|
|
|
|
|
## TorDNSEL
|
|
|
|
|
|
TorDNSEL is a DNS list service that behaves in a similar way to [Domain Name System-based Blackhole Lists](https://en.wikipedia.org/wiki/Domain_Name_System-based_Blackhole_List).
|
|
|
IP addresses will give positive results in the event that an address has been found to be used by an exit relay in a recent scan.
|
|
|
|
|
|
As with the exit lists, IP addresses will be removed after not being seen for
|
|
|
48 hours.
|
|
|
|
|
|
A future update to this script may include information from the consensus, but
|
|
|
currently a postitive match must have been found from the exit scanner for an
|
|
|
IP address to be included.
|
|
|
|
|
|
## Tor Check
|
|
|
|
|
|
Tor Check is a website that can be used to determine if a browser is using the Tor network for queries.
|
|
|
It will also check the User-Agent to determine if a user is using Tor Browser.
|
|
|
It is accessed via HTTPS at <https://check.torproject.org/>.
|
|
|
|
|
|
There are also JSON APIs:
|
|
|
|
|
|
- <https://check.torproject.org/api/ip> - Check IP and if it's a Tor IP
|
|
|
- <https://check.torproject.org/api/bulk> - Bulk exit list exporter
|
|
|
|
|
|
# Contacts
|
|
|
|
|
|
The primary contact for this service is the Metrics Team <[metrics-team@lists.torproject.org](mailto:metrics-team@lists.torproject.org)>.
|
|
|
For urgent queries, contact **karsten** or **gaba** in IRC.
|
|
|
|
|
|
# Implementation Specifics
|
|
|
|
|
|
The underlying infrastructure for the exit scanner, TorDNSEL and Tor Check services is provided by the
|
|
|
Tor Sysadmin Team (TSA). All services run on one virtual machine with the hostname `check-01.torproject.org`.
|
|
|
|
|
|
## Exit Scanner
|
|
|
|
|
|
The exitmap module used is a custom module named [ipscan](https://gitweb.torproject.org/metrics-cloud.git/tree/ansible/roles/exit-scanner/files/ipscan.py).
|
|
|
|
|
|
The
|
|
|
[exitscan](https://gitweb.torproject.org/metrics-cloud.git/tree/ansible/roles/exit-scanner/files/exitscan.py)
|
|
|
wrapper runs as a systemd user service under the `tordnsel` user, named
|
|
|
[exitscanner.service](https://gitweb.torproject.org/metrics-cloud.git/tree/ansible/roles/exit-scanner/files/exitscanner.service).
|
|
|
|
|
|
## TorDNSEL
|
|
|
|
|
|
The exitscan wrapper produces a BIND zone file
|
|
|
`/srv/tordnsel.torproject.org/dnsel.torproject.org`. The wrapper also
|
|
|
automatically reloads this zone file in the TPA managed BIND server whenever it
|
|
|
has been updated. The name `dnsel.torproject.org` is delegated to the `check-01`
|
|
|
host with NS records.
|
|
|
|
|
|
weasel had ideas about how we could run this as a hidden master, transfer the
|
|
|
zone to the main domain servers and also have them sign the zone with DNSSEC.
|
|
|
This is an idea for the future.
|
|
|
|
|
|
# Sources
|
|
|
|
|
|
The sources for exitmap are available on GitHub: <https://github.com/NullHypothesis/exitmap>.
|
|
|
The [exitmap wrapper](https://gitweb.torproject.org/metrics-cloud.git/tree/ansible/roles/exit-scanner/files/exitscan.py) and [module](https://gitweb.torproject.org/metrics-cloud.git/tree/ansible/roles/exit-scanner/files/ipscan.py) used by the exit scanner can be found in the metrics-cloud repository.
|
|
|
|
|
|
The wrapper script is also responsible for writing out the zone file to be used by the TorDNSEL service
|
|
|
and triggering a reload of the zone.
|
|
|
|
|
|
The sources for Tor Check are available in our git: <https://gitweb.torproject.org/check.git>.
|
|
|
|
|
|
# Deployment
|
|
|
|
|
|
## Initial deployment
|
|
|
|
|
|
The initial deployment procedure is split into 2 parts:
|
|
|
|
|
|
- Installing and starting the services
|
|
|
|
|
|
There are no manual steps required to load state, and backups do not need to be
|
|
|
performed for the host running this service.
|
|
|
|
|
|
Everything can be configured from scratch with only the Ansible playbook.
|
|
|
|
|
|
### Development/testing in AWS
|
|
|
|
|
|
For development or testing in AWS, a CloudFormation template is available named [`exit-scanner-dev.yml`](https://gitweb.torproject.org/metrics-cloud.git/plain/cloudformation/exit-scanner-dev.yml).
|
|
|
|
|
|
From the [CloudFormation portal](https://console.aws.amazon.com/cloudformation/home?region=us-east-1#/stacks), select your stack and view the outputs.
|
|
|
You will find here the public IP address for the EC2 instance that has been created.
|
|
|
Add this instance to **ansible/dev** in your local copy of metrics-cloud.git under "[exit-scanners]".
|
|
|
|
|
|
You can now setup the machine with Ansible by running:
|
|
|
|
|
|
```
|
|
|
ansible-playbook -i dev exit-scanners-aws.yml
|
|
|
```
|
|
|
|
|
|
Note that the AWS AMI used has passwordless sudo, so no password need be given.
|
|
|
|
|
|
### Fresh machine from TSA
|
|
|
|
|
|
Add the host name of the new instance to **ansible/production** in your local
|
|
|
copy of metrics-cloud.git under "[exit-scanners]" and commit the change.
|
|
|
|
|
|
You can now setup the machine with Ansible by running:
|
|
|
|
|
|
```
|
|
|
ansible-playbook -i production -K exit-scanners.yml
|
|
|
```
|
|
|
|
|
|
## Upgrade
|
|
|
|
|
|
The upstream sources for the applications that make up this service do not have managed releases
|
|
|
which makes this difficult.
|
|
|
|
|
|
To fix a bug in the exit scanner wrapper script, fix this in the metrics-cloud repository and re-run
|
|
|
the deployment playbook.
|
|
|
|
|
|
- [ ] Can we upgrade exitmap sensibly?
|
|
|
- [ ] Can we upgrade Tor Check sensibly?
|
|
|
|
|
|
# Diagnostics
|
|
|
|
|
|
## Logs
|
|
|
|
|
|
The exit scanner wrapper logs output to `/srv/tordnsel.torproject.org/exitscanner.log`.
|
|
|
|
|
|
These logs include every measurement ever performed, and this might fill up the disk eventually. See #33475.
|
|
|
|
|
|
# Monitoring
|
|
|
|
|
|
CollecTor gives a warning in the case that the exit lists are too old.
|
|
|
|
|
|
There are no Nagios checks for this service beyond the standard TPA systems checks (and I didn't actually check if we have those yet).
|
|
|
|
|
|
# Disaster Recovery
|
|
|
|
|
|
## Service needs restart
|
|
|
|
|
|
The service can be restarted, as the tordnsel user, with:
|
|
|
|
|
|
```
|
|
|
XDG_RUNTIME_DIR=/run/user/`id -u` systemctl --user restart exitscanner.service
|
|
|
```
|
|
|
|
|
|
## Total Loss
|
|
|
|
|
|
The exit scanner service does not need to maintain any state between runs.
|
|
|
It's nice if it can in order to cope with a relay that happened to be down at the time we tried to measure
|
|
|
it but in the event of a failure it is perfectly acceptable to throw away the old box and provision a new one.
|
|
|
Follow the initial deployment instructions above.
|
|
|
|
|
|
# Standards
|
|
|
|
|
|
The exit scanner service produces exit lists according to the [TorDNSEL exit list format](https://2019.www.torproject.org/tordnsel/exitlist-spec.txt).
|
|
|
|
|
|
# Major Caveats
|
|
|
|
|
|
The exit scanner service does not support IPv6.
|
|
|
|
|
|
Serial numbers for the DNS zones are based on the date. This needs to be better
|
|
|
planned out before 2042, where we will see integer overflows on the serial
|
|
|
number.
|
|
|
|
|
|
Previous exit lists to merge are found using a glob that will break after 23:59
|
|
|
on 31st Dec 2999.
|
|
|
|
|
|
# Bugs
|
|
|
|
|
|
Known bugs can be found in the Tor Project Trac for:
|
|
|
|
|
|
- [Exit Scanner](https://gitlab.torproject.org/tpo/metrics/exit-scanner/-/issues)
|
|
|
- [Tor Check](https://gitlab.torproject.org/tpo/web/tor-check/-/issues)
|
|
|
|
|
|
For bugs relating to exitmap, they are found on the GitHub project: <https://github.com/NullHypothesis/exitmap/issues>
|
|
|
|
|
|
New bug reports should be filed in the appropriate tracker and component.
|
|
|
|