Commit 6d05ae95 authored by Karsten Loesing's avatar Karsten Loesing
Browse files

Document all public parts using Javadoc.

Based in parts on very helpful suggestions and feedback by iwakeh.

Implements #16873.
parent 1413dfb1
......@@ -12,6 +12,8 @@
implementation classes.
- Actually return the signing key digest in network status votes.
- Parse crypto parts in network status votes.
- Document all public parts in org.torproject.descriptor and add
an Ant target to generate Javadocs.
* Minor changes
- Include a Torperf results line with more than one unrecognized
......
<project default="jar" name="descriptor" basedir=".">
<property name="release.version" value="1.1.0-dev" />
<property name="sources" value="src"/>
<property name="resources" value="resources"/>
<property name="classes" value="classes"/>
<property name="docs" value="javadoc"/>
<property name="tests" value="test"/>
<property name="libs" value="lib"/>
<property name="jarfile" value="descriptor-${release.version}.jar" />
......@@ -25,6 +27,7 @@
<target name="clean" >
<delete includeEmptyDirs="true">
<fileset dir="${classes}" defaultexcludes="false" includes="**" />
<fileset dir="${docs}" defaultexcludes="false" includes="**" />
</delete>
<delete file="${jarfile}"/>
<delete file="${jarsourcesfile}"/>
......@@ -33,6 +36,7 @@
<target name="init">
<mkdir dir="${classes}"/>
<mkdir dir="${docs}"/>
</target>
<target name="compile"
......@@ -50,6 +54,20 @@
</javac>
</target>
<target name="docs">
<javadoc destdir="${docs}"
footer="&amp;copy; 2016 The Tor Project"
doctitle="DescripTor API Documentation"
overview="${basedir}/${resources}/overview.html"
use="true"
windowtitle="DescripTor API Documentation">
<classpath refid="classpath"/>
<fileset dir="${sources}">
<exclude name="**/impl/**"/>
</fileset>
</javadoc>
</target>
<target name="test" depends="compile">
<javac destdir="${classes}"
srcdir="${tests}"
......
<html>
<body>
<p>DescripTor API, which is provided and supported by Tor's Metrics
Team, is a library to obtain and process descriptors containing Tor
network data. It is the main Java tool for processing Tor descriptors
and provides a standard API consisting of interfaces and a reference
implementation for all of them.</p>
<p>Most Tor descriptors understood by this library are specified in the
<a href="https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt">Tor
directory protocol, version 3</a> or in the earlier
<a href="https://gitweb.torproject.org/torspec.git/tree/attic/dir-spec-v2.txt">version 2</a> or
<a href="https://gitweb.torproject.org/torspec.git/tree/attic/dir-spec-v1.txt">version 1</a>
of that document.
Other descriptors are specified on the
<a href="https://collector.torproject.org/">CollecTor website</a>.</p>
<p>The interfaces in
<a href="./org/torproject/descriptor/package-summary.html">{@code org.torproject.descriptor}</a>
as well as their implementations in the
{@code org.torproject.descriptor.impl} package were driven by two main
goals originating from the primary use case to make Tor network data
accessible for statistical analysis:</p>
<ul>
<li><em>Complete coverage:</em> This library is supposed to cover the
complete range of Tor descriptors made available by the
<a href="https://collector.torproject.org">CollecTor</a> service.</li>
<li><em>Runtime and memory efficiency:</em> Processing large amounts of
descriptors in bulk is supposed to be efficient in terms of runtime and
required memory.</li>
</ul>
<p>At the same time the current design and implementation were done with a
number of non-goals in mind, even though some of these might turn into
goals in the future:</p>
<p><ul>
<li><em>Verification:</em> The descriptor parser performs some basic
verifications of descriptor formats, but no cryptographic verifications.
It may not even be possible to write a cryptographic verification tool
using parsed descriptor contents, though this has not been attempted
yet.</li>
<li><em>Potentially lossy conversion:</em> Descriptor contents may be
converted to a format that is easier to process, even if that conversion
makes it harder or impossible to re-create the original descriptor
contents from a parsed descriptor.</li>
<li><em>Generating descriptors:</em> This library does not contain any
functionality to generate new descriptors for testing or related purposes,
neither from previously set data nor randomly.</li>
<li><em>Writing descriptors:</em> This library does not support writing
descriptors to the file system or a database, both of which are left to
the application. Stated differently, there are no descriptor sinks that
would correspond to the provided descriptor sources.</li>
</ul></p>
<p>Hints about using DescripTor can be found in the
<a href="./org/torproject/descriptor/package-summary.html#package.description">{@code org.torproject.descriptor} package description</a>.
</p>
<p>Contact and further information:
<ul>
<li><a href="https://trac.torproject.org/projects/tor/query?status=!closed&component=Metrics%2Fmetrics-lib">DescripTor Bug Tracker</a></li>
<li><a href="https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team">Metrics Team Mailing List</a></li>
<li><a href="https://trac.torproject.org/projects/tor/wiki/org/teams/MetricsTeam">Metrics Team Website</a></li>
</ul>
</body>
</html>
/* Copyright 2012--2015 The Tor Project
/* Copyright 2012--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
import java.util.SortedMap;
/* Contains the bandwidth history of a relay or bridge. */
/**
* Contains the bandwidth history of a relay or bridge.
*
* <p>A bandwidth history is not a descriptor type of its own but usually
* part of extra-info descriptors ({@link ExtraInfoDescriptor}) or server
* descriptors ({@link ServerDescriptor}).</p>
*
* @since 1.0.0
*/
public interface BandwidthHistory {
/* Return the original bandwidth history line as contained in the
* descriptor, possibly prefixed with "opt ". */
/**
* Return the original bandwidth history line as contained in the
* descriptor, possibly prefixed with {@code "opt "}.
*
* @since 1.0.0
*/
public String getLine();
/* Return the end of the most recent interval in millis. */
/**
* Return the time in milliseconds since the epoch when the most recent
* interval ends.
*
* @since 1.0.0
*/
public long getHistoryEndMillis();
/* Return the interval length in seconds, which is typically 900 seconds
* or 15 minutes. */
/**
* Return the interval length in seconds.
*
* @since 1.0.0
*/
public long getIntervalLength();
/* Return the (possibly empty) bandwidth history with map keys being
* interval ends in millis and map values being number of bytes used in
* the interval, ordered from oldest to newest interval. */
/**
* Return the (possibly empty) bandwidth history with map keys being
* times in milliseconds since the epoch when intervals end and map
* values being number of bytes used in the interval, ordered from
* oldest to newest interval.
*
* @since 1.0.0
*/
public SortedMap<Long, Long> getBandwidthValues();
}
/* Copyright 2015 The Tor Project
/* Copyright 2015--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
/* Contains a bridge extra-info descriptor. */
/**
* Contains a sanitized bridge extra-info descriptor.
*
* <p>Sanitized bridge extra-info descriptors share many contents with
* relay extra-info descriptors ({@link RelayExtraInfoDescriptor}), which
* is why they share a common
* superinterface ({@link ExtraInfoDescriptor}). The main purpose of
* having two subinterfaces is being able to distinguish descriptor types
* more easily.</p>
*
* <p>Details about sanitizing bridge extra-info descriptors can be found
* <a href="https://collector.torproject.org/#type-bridge-extra-info">here</a>.
* </p>
*
* @since 1.1.0
*/
public interface BridgeExtraInfoDescriptor extends ExtraInfoDescriptor {
}
......
/* Copyright 2011--2015 The Tor Project
/* Copyright 2011--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
import java.util.SortedMap;
/**
* Contains a sanitized bridge network status document.
*
* <p>The bridge directory authority periodically publishes a network
* status document with one entry per known bridge in the network
* ({@link NetworkStatusEntry}) containing: a hash of its identity key, a
* hash of its most recent server descriptor, and a summary of what the
* bridge authority believed about its status.</p>
*
* <p>The main purpose of this document is to get an authoritative list of
* running bridges to the bridge distribution service BridgeDB.</p>
*
* <p>Details about sanitizing bridge network statuses can be found
* <a href="https://collector.torproject.org/#type-bridge-network-status">here</a>.
* </p>
*
* @since 1.0.0
*/
public interface BridgeNetworkStatus extends Descriptor {
/* Return the published time in milliseconds. */
/**
* Return the time in milliseconds since the epoch when this descriptor
* was published.
*
* @since 1.0.0
*/
public long getPublishedMillis();
/* Return the minimum uptime in seconds that this authority requires for
* assigning the Stable flag, or -1 if the authority doesn't report this
* value. */
/**
* Return the minimum uptime in seconds that this authority requires
* for assigning the Stable flag, or -1 if the authority doesn't report
* this value.
*
* @since 1.1.0
*/
public long getStableUptime();
/* Return the minimum MTBF (mean time between failure) that this
/**
* Return the minimum MTBF (mean time between failure) that this
* authority requires for assigning the Stable flag, or -1 if the
* authority doesn't report this value. */
* authority doesn't report this value.
*
* @since 1.1.0
*/
public long getStableMtbf();
/* Return the minimum bandwidth that this authority requires for
/**
* Return the minimum bandwidth that this authority requires for
* assigning the Fast flag, or -1 if the authority doesn't report this
* value. */
* value.
*
* @since 1.1.0
*/
public long getFastBandwidth();
/* Return the minimum WFU (weighted fractional uptime) in percent that
* this authority requires for assigning the Guard flag, or -1.0 if the
* authority doesn't report this value. */
/**
* Return the minimum WFU (weighted fractional uptime) in percent that
* this authority requires for assigning the Guard flag, or -1 if the
* authority doesn't report this value.
*
* @since 1.1.0
*/
public double getGuardWfu();
/* Return the minimum weighted time in seconds that this authority needs
* to know about a relay before assigning the Guard flag, or -1 if the
* authority doesn't report this information. */
/**
* Return the minimum weighted time in seconds that this authority
* needs to know about a relay before assigning the Guard flag, or -1 if
* the authority doesn't report this information.
*
* @since 1.1.0
*/
public long getGuardTk();
/* Return the minimum bandwidth that this authority requires for
/**
* Return the minimum bandwidth that this authority requires for
* assigning the Guard flag if exits can be guards, or -1 if the
* authority doesn't report this value. */
* authority doesn't report this value.
*
* @since 1.1.0
*/
public long getGuardBandwidthIncludingExits();
/* Return the minimum bandwidth that this authority requires for
/**
* Return the minimum bandwidth that this authority requires for
* assigning the Guard flag if exits can not be guards, or -1 if the
* authority doesn't report this value. */
* authority doesn't report this value.
*
* @since 1.1.0
*/
public long getGuardBandwidthExcludingExits();
/* Return 1 if the authority has measured enough MTBF info to use the
/**
* Return 1 if the authority has measured enough MTBF info to use the
* MTBF requirement instead of the uptime requirement for assigning the
* Stable flag, 0 if not, or -1 if the authority doesn't report this
* information. */
* information.
*
* @since 1.1.0
*/
public int getEnoughMtbfInfo();
/* Return 1 if the authority has enough measured bandwidths that it'll
/**
* Return 1 if the authority has enough measured bandwidths that it'll
* ignore the advertised bandwidth claims of routers without measured
* bandwidth, 0 if not, or -1 if the authority doesn't report this
* information. */
* information.
*
* @since 1.1.0
*/
public int getIgnoringAdvertisedBws();
/* Return status entries, one for each contained bridge. */
/**
* Return status entries for each contained bridge, with map keys being
* SHA-1 digests of SHA-1 digest of the bridges' public identity keys,
* encoded as 40 upper-case hexadecimal characters.
*
* @since 1.0.0
*/
public SortedMap<String, NetworkStatusEntry> getStatusEntries();
}
/* Copyright 2012--2015 The Tor Project
/* Copyright 2012--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
import java.util.SortedMap;
/**
* Contains a sanitized list of bridges together with the distribution
* pools they have been assigned to by the bridge distribution service
* BridgeDB.
*
* <p>BridgeDB receives bridge network statuses
* ({@link BridgeNetworkStatus}) from the bridge authority, assigns these
* bridges to persistent distribution rings, and hands them out to bridge
* users. BridgeDB periodically dumps the list of running bridges with
* information about the rings, subrings, and file buckets to which they
* are assigned to a local file.</p>
*
* <p>Details about sanitizing bridge pool assignments can be found
* <a href="https://collector.torproject.org/#type-bridge-pool-assignment">here</a>.
* </p>
*
* @since 1.0.0
*/
public interface BridgePoolAssignment extends Descriptor {
/* Return the publication time of this bridge pool assignment list. */
/**
* Return the time in milliseconds since the epoch when this descriptor
* was published.
*
* @since 1.0.0
*/
public long getPublishedMillis();
/* Return the entries contained in this bridge pool assignment list with
* map keys being bridge fingerprints and map values being assignment
* strings, e.g. "https ring=3 flag=stable". */
/**
* Return the entries contained in this bridge pool assignment list
* with map keys being SHA-1 digests of SHA-1 digest of the bridges'
* public identity keys, encoded as 40 upper-case hexadecimal
* characters, and map values being assignment strings, e.g.
* {@code "https ring=3 flag=stable"}.
*
* @since 1.0.0
*/
public SortedMap<String, String> getEntries();
}
/* Copyright 2015 The Tor Project
/* Copyright 2015--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
/* Contains a bridge server descriptor. */
/**
* Contains a sanitized bridge server descriptor.
*
* <p>Sanitized bridge server descriptors share many contents with relay
* server descriptors ({@link RelayServerDescriptor}), which is why they
* share a common superinterface ({@link ServerDescriptor}). The main
* purpose of having two subinterfaces is being able to distinguish
* descriptor types more easily.</p>
*
* <p>Details about sanitizing bridge server descriptors can be found
* <a href="https://collector.torproject.org/#type-bridge-server-descriptor">here</a>.
* </p>
*
* @since 1.1.0
*/
public interface BridgeServerDescriptor extends ServerDescriptor {
}
......
/* Copyright 2011--2015 The Tor Project
/* Copyright 2011--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
import java.util.List;
/* Store meta-data about how a descriptor was downloaded or read from
* disk. */
/**
* Superinterface for any descriptor with access to generic information
* about the descriptor.
*
* @since 1.0.0
*/
public interface Descriptor {
/* Return the raw descriptor bytes. */
/**
* Return the raw descriptor bytes.
*
* @since 1.0.0
*/
public byte[] getRawDescriptorBytes();
/* Return the (possibly empty) list of annotations. */
/**
* Return the (possibly empty) list of annotations in the format
* {@code "@key( value)*"}.
*
* @since 1.0.0
*/
public List<String> getAnnotations();
/* Return any unrecognized lines when parsing this descriptor, or an
* empty list if there were no unrecognized lines. */
/**
* Return any unrecognized lines when parsing this descriptor, or an
* empty list if there were no unrecognized lines.
*
* @since 1.0.0
*/
public List<String> getUnrecognizedLines();
}
/* Copyright 2015 The Tor Project
/* Copyright 2015--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
import java.io.File;
/** Fetch descriptors from the CollecTor service available at
* https://collector.torproject.org/ and store them to a local
* directory. */
/**
* Descriptor source that synchronizes descriptors from the CollecTor
* service to a given local directory.
*
* <p>This type is not a descriptor source in the proper sense, because it
* does not produce descriptors by itself. But it often creates the
* prerequisites for reading descriptors from disk using
* {@link DescriptorReader}.</p>
*
* <p>Code sample:</p>
* <pre>{@code
* DescriptorCollector descriptorCollector =
* DescriptorSourceFactory.createDescriptorCollector();
* descriptorCollector.collectDescriptors(
* // Download from Tor's main CollecTor instance,
* "https://collector.torproject.org",
* // include network status consensuses and relay server descriptors
* new String[] { "/recent/relay-descriptors/consensuses/",
* "/recent/relay-descriptors/server-descriptors/" },
* // regardless of last-modified time,
* 0L,
* // write to the local directory called in/,
* new File("in"),
* // and delete extraneous files that do not exist remotely anymore.
* true);
* }</pre>
*
* @since 1.0.0
*/
public interface DescriptorCollector {
/**
......@@ -15,17 +42,18 @@ public interface DescriptorCollector {
* anymore.
*
* @param collecTorBaseUrl CollecTor base URL without trailing slash,
* e.g., "https://collector.torproject.org".
* e.g., {@code "https://collector.torproject.org"}
* @param remoteDirectories Remote directories to collect descriptors
* from, e.g., "/recent/relay-descriptors/server-descriptors/". Only
* files in this directory will be collected, no files in its sub
* directories.
* from, e.g.,
* {@code "/recent/relay-descriptors/server-descriptors/"}, without
* processing subdirectories unless they are explicitly listed
* @param minLastModified Minimum last-modified time in milliseconds of
* files to be collected. Set to 0 for collecting all files.
* @param localDirectory Directory where collected files will be
* written.
* files to be collected, or 0 for collecting all files
* @param localDirectory Directory where collected files will be written
* @param deleteExtraneousLocalFiles Whether to delete all local files
* that do not exist remotely anymore.
* that do not exist remotely anymore
*
* @since 1.0.0
*/
public void collectDescriptors(String collecTorBaseUrl,
String[] remoteDirectories, long minLastModified,
......
/* Copyright 2011--2015 The Tor Project
/* Copyright 2011--2016 The Tor Project
* See LICENSE for licensing information */
package org.torproject.descriptor;
import java.util.Iterator;
import java.util.Set;
/* Download relay descriptors from directory mirrors or authorities. */
/**
* Descriptor source that downloads relay descriptors from directory
* authorities or mirrors.
*
* <p>Downloading descriptors is done in a batch which starts after
* setting any configuration options and initiating the download
* process.</p>
*
* @since 1.0.0
*/
public interface DescriptorDownloader {
/* Add a directory authority to download descriptors from. A directory
* authority is only required for downloading network status vote and
* will be used when no directory mirrors are available. */
/**
* Add a directory authority to download descriptors from, which is
* only required for downloading network status votes and will be used
* when no directory mirrors are available.
*
* @since 1.0.0
*/
public void addDirectoryAuthority(String nickname, String ip,
int dirPort);
/* Add a directory mirror to download descriptors from. Directory
* mirrors are preferred when downloading descriptors, except for
* network status votes which are only available on directory
* authorities. */
/**
* Add a directory mirror to download descriptors from, which is
* preferred for downloading descriptors, except for network status
* votes which are only available on directory authorities.
*
* @since 1.0.0
*/
public void addDirectoryMirror(String nickname, String ip, int dirPort);
/* Include the current network status consensus in the downloads. */
/**