Document all public parts using Javadoc.

Based in parts on very helpful suggestions and feedback by iwakeh. Implements #16873.

Document all public parts using Javadoc.
Based in parts on very helpful suggestions and feedback by iwakeh. Implements #16873.
6d05ae95 · Karsten Loesing · 1413dfb1 · 6d05ae95 · 6d05ae95 · 6d05ae95
Commit 6d05ae95 authored May 20, 2016 by Karsten Loesing
--- a/CHANGELOG.md
+++ b/CHANGELOG.md
@@ -12,6 +12,8 @@
     implementation classes.
   - Actually return the signing key digest in network status votes.
   - Parse crypto parts in network status votes.
+   - Document all public parts in org.torproject.descriptor and add
+     an Ant target to generate Javadocs.

 * Minor changes
   - Include a Torperf results line with more than one unrecognized

--- a/build.xml
+++ b/build.xml
 <project default="jar" name="descriptor" basedir=".">
  <property name="release.version" value="1.1.0-dev" />
  <property name="sources" value="src"/>
+  <property name="resources" value="resources"/>
  <property name="classes" value="classes"/>
+  <property name="docs" value="javadoc"/>
  <property name="tests" value="test"/>
  <property name="libs" value="lib"/>
  <property name="jarfile" value="descriptor-${release.version}.jar" />
@@ -25,6 +27,7 @@
  <target name="clean" >
    <delete includeEmptyDirs="true">
      <fileset dir="${classes}" defaultexcludes="false" includes="**" />
+      <fileset dir="${docs}" defaultexcludes="false" includes="**" />
    </delete>
    <delete file="${jarfile}"/>
    <delete file="${jarsourcesfile}"/>
@@ -33,6 +36,7 @@

  <target name="init">
    <mkdir dir="${classes}"/>
+    <mkdir dir="${docs}"/>
  </target>

  <target name="compile"
@@ -50,6 +54,20 @@
    </javac>
  </target>

+  <target name="docs">
+    <javadoc destdir="${docs}"
+             footer="&amp;copy; 2016 The Tor Project"
+             doctitle="DescripTor API Documentation"
+             overview="${basedir}/${resources}/overview.html"
+             use="true"
+             windowtitle="DescripTor API Documentation">
+      <classpath refid="classpath"/>
+      <fileset dir="${sources}">
+        <exclude name="**/impl/**"/>
+      </fileset>
+    </javadoc>
+  </target>
+
  <target name="test" depends="compile">
    <javac destdir="${classes}"
           srcdir="${tests}"

--- a/resources/overview.html
+++ b/resources/overview.html
+<html>
+<body>
+<p>DescripTor API, which is provided and supported by Tor's Metrics
+Team, is a library to obtain and process descriptors containing Tor
+network data.  It is the main Java tool for processing Tor descriptors
+and provides a standard API consisting of interfaces and a reference
+implementation for all of them.</p>
+
+<p>Most Tor descriptors understood by this library are specified in the
+<a href="https://gitweb.torproject.org/torspec.git/tree/dir-spec.txt">Tor
+directory protocol, version 3</a> or in the earlier
+<a href="https://gitweb.torproject.org/torspec.git/tree/attic/dir-spec-v2.txt">version 2</a> or
+<a href="https://gitweb.torproject.org/torspec.git/tree/attic/dir-spec-v1.txt">version 1</a>
+of that document.
+Other descriptors are specified on the
+<a href="https://collector.torproject.org/">CollecTor website</a>.</p>
+
+<p>The interfaces in
+<a href="./org/torproject/descriptor/package-summary.html">{@code org.torproject.descriptor}</a>
+as well as their implementations in the
+{@code org.torproject.descriptor.impl} package were driven by two main
+goals originating from the primary use case to make Tor network data
+accessible for statistical analysis:</p>
+
+<ul>
+<li><em>Complete coverage:</em> This library is supposed to cover the
+complete range of Tor descriptors made available by the
+<a href="https://collector.torproject.org">CollecTor</a> service.</li>
+<li><em>Runtime and memory efficiency:</em> Processing large amounts of
+descriptors in bulk is supposed to be efficient in terms of runtime and
+required memory.</li>
+</ul>
+
+<p>At the same time the current design and implementation were done with a
+number of non-goals in mind, even though some of these might turn into
+goals in the future:</p>
+
+<p><ul>
+<li><em>Verification:</em> The descriptor parser performs some basic
+verifications of descriptor formats, but no cryptographic verifications.
+It may not even be possible to write a cryptographic verification tool
+using parsed descriptor contents, though this has not been attempted
+yet.</li>
+<li><em>Potentially lossy conversion:</em> Descriptor contents may be
+converted to a format that is easier to process, even if that conversion
+makes it harder or impossible to re-create the original descriptor
+contents from a parsed descriptor.</li>
+<li><em>Generating descriptors:</em> This library does not contain any
+functionality to generate new descriptors for testing or related purposes,
+neither from previously set data nor randomly.</li>
+<li><em>Writing descriptors:</em> This library does not support writing
+descriptors to the file system or a database, both of which are left to
+the application.  Stated differently, there are no descriptor sinks that
+would correspond to the provided descriptor sources.</li>
+</ul></p>
+
+<p>Hints about using DescripTor can be found in the
+<a href="./org/torproject/descriptor/package-summary.html#package.description">{@code org.torproject.descriptor} package description</a>.
+</p>
+
+<p>Contact and further information:
+<ul>
+<li><a href="https://trac.torproject.org/projects/tor/query?status=!closed&component=Metrics%2Fmetrics-lib">DescripTor Bug Tracker</a></li>
+<li><a href="https://lists.torproject.org/cgi-bin/mailman/listinfo/metrics-team">Metrics Team Mailing List</a></li>
+<li><a href="https://trac.torproject.org/projects/tor/wiki/org/teams/MetricsTeam">Metrics Team Website</a></li>
+</ul>
+</body>
+</html>
+
--- a/src/org/torproject/descriptor/BandwidthHistory.java
+++ b/src/org/torproject/descriptor/BandwidthHistory.java
-/* Copyright 2012--2015 The Tor Project
+/* Copyright 2012--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

 import java.util.SortedMap;

-/* Contains the bandwidth history of a relay or bridge. */
+/**
+ * Contains the bandwidth history of a relay or bridge.
+ *
+ * <p>A bandwidth history is not a descriptor type of its own but usually
+ * part of extra-info descriptors ({@link ExtraInfoDescriptor}) or server
+ * descriptors ({@link ServerDescriptor}).</p>
+ *
+ * @since 1.0.0
+ */
 public interface BandwidthHistory {

-  /* Return the original bandwidth history line as contained in the
-   * descriptor, possibly prefixed with "opt ". */
+  /**
+   * Return the original bandwidth history line as contained in the
+   * descriptor, possibly prefixed with {@code "opt "}.
+   *
+   * @since 1.0.0
+   */
  public String getLine();

-  /* Return the end of the most recent interval in millis. */
+  /**
+   * Return the time in milliseconds since the epoch when the most recent
+   * interval ends.
+   *
+   * @since 1.0.0
+   */
  public long getHistoryEndMillis();

-  /* Return the interval length in seconds, which is typically 900 seconds
-   * or 15 minutes. */
+  /**
+   * Return the interval length in seconds.
+   *
+   * @since 1.0.0
+   */
  public long getIntervalLength();

-  /* Return the (possibly empty) bandwidth history with map keys being
-   * interval ends in millis and map values being number of bytes used in
-   * the interval, ordered from oldest to newest interval. */
+  /**
+   * Return the (possibly empty) bandwidth history with map keys being
+   * times in milliseconds since the epoch when intervals end and map
+   * values being number of bytes used in the interval, ordered from
+   * oldest to newest interval.
+   *
+   * @since 1.0.0
+   */
  public SortedMap<Long, Long> getBandwidthValues();
 }

--- a/src/org/torproject/descriptor/BridgeExtraInfoDescriptor.java
+++ b/src/org/torproject/descriptor/BridgeExtraInfoDescriptor.java
-/* Copyright 2015 The Tor Project
+/* Copyright 2015--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

-/* Contains a bridge extra-info descriptor. */
+/**
+ * Contains a sanitized bridge extra-info descriptor.
+ *
+ * <p>Sanitized bridge extra-info descriptors share many contents with
+ * relay extra-info descriptors ({@link RelayExtraInfoDescriptor}), which
+ * is why they share a common
+ * superinterface ({@link ExtraInfoDescriptor}).  The main purpose of
+ * having two subinterfaces is being able to distinguish descriptor types
+ * more easily.</p>
+ *
+ * <p>Details about sanitizing bridge extra-info descriptors can be found
+ * <a href="https://collector.torproject.org/#type-bridge-extra-info">here</a>.
+ * </p>
+ *
+ * @since 1.1.0
+ */
 public interface BridgeExtraInfoDescriptor extends ExtraInfoDescriptor {

 }

--- a/src/org/torproject/descriptor/BridgeNetworkStatus.java
+++ b/src/org/torproject/descriptor/BridgeNetworkStatus.java
-/* Copyright 2011--2015 The Tor Project
+/* Copyright 2011--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

 import java.util.SortedMap;

+/**
+ * Contains a sanitized bridge network status document.
+ *
+ * <p>The bridge directory authority periodically publishes a network
+ * status document with one entry per known bridge in the network
+ * ({@link NetworkStatusEntry}) containing: a hash of its identity key, a
+ * hash of its most recent server descriptor, and a summary of what the
+ * bridge authority believed about its status.</p>
+ *
+ * <p>The main purpose of this document is to get an authoritative list of
+ * running bridges to the bridge distribution service BridgeDB.</p>
+ *
+ * <p>Details about sanitizing bridge network statuses can be found
+ * <a href="https://collector.torproject.org/#type-bridge-network-status">here</a>.
+ * </p>
+ *
+ * @since 1.0.0
+ */
 public interface BridgeNetworkStatus extends Descriptor {

-  /* Return the published time in milliseconds. */
+  /**
+   * Return the time in milliseconds since the epoch when this descriptor
+   * was published.
+   *
+   * @since 1.0.0
+   */
  public long getPublishedMillis();

-  /* Return the minimum uptime in seconds that this authority requires for
-   * assigning the Stable flag, or -1 if the authority doesn't report this
-   * value. */
+  /**
+   * Return the minimum uptime in seconds that this authority requires
+   * for assigning the Stable flag, or -1 if the authority doesn't report
+   * this value.
+   *
+   * @since 1.1.0
+   */
  public long getStableUptime();

-  /* Return the minimum MTBF (mean time between failure) that this
+  /**
+   * Return the minimum MTBF (mean time between failure) that this
   * authority requires for assigning the Stable flag, or -1 if the
-   * authority doesn't report this value. */
+   * authority doesn't report this value.
+   *
+   * @since 1.1.0
+   */
  public long getStableMtbf();

-  /* Return the minimum bandwidth that this authority requires for
+  /**
+   * Return the minimum bandwidth that this authority requires for
   * assigning the Fast flag, or -1 if the authority doesn't report this
-   * value. */
+   * value.
+   *
+   * @since 1.1.0
+   */
  public long getFastBandwidth();

-  /* Return the minimum WFU (weighted fractional uptime) in percent that
-   * this authority requires for assigning the Guard flag, or -1.0 if the
-   * authority doesn't report this value. */
+  /**
+   * Return the minimum WFU (weighted fractional uptime) in percent that
+   * this authority requires for assigning the Guard flag, or -1 if the
+   * authority doesn't report this value.
+   *
+   * @since 1.1.0
+   */
  public double getGuardWfu();

-  /* Return the minimum weighted time in seconds that this authority needs
-   * to know about a relay before assigning the Guard flag, or -1 if the
-   * authority doesn't report this information. */
+  /**
+   * Return the minimum weighted time in seconds that this authority
+   * needs to know about a relay before assigning the Guard flag, or -1 if
+   * the authority doesn't report this information.
+   *
+   * @since 1.1.0
+   */
  public long getGuardTk();

-  /* Return the minimum bandwidth that this authority requires for
+  /**
+   * Return the minimum bandwidth that this authority requires for
   * assigning the Guard flag if exits can be guards, or -1 if the
-   * authority doesn't report this value. */
+   * authority doesn't report this value.
+   *
+   * @since 1.1.0
+   */
  public long getGuardBandwidthIncludingExits();

-  /* Return the minimum bandwidth that this authority requires for
+  /**
+   * Return the minimum bandwidth that this authority requires for
   * assigning the Guard flag if exits can not be guards, or -1 if the
-   * authority doesn't report this value. */
+   * authority doesn't report this value.
+   *
+   * @since 1.1.0
+   */
  public long getGuardBandwidthExcludingExits();

-  /* Return 1 if the authority has measured enough MTBF info to use the
+  /**
+   * Return 1 if the authority has measured enough MTBF info to use the
   * MTBF requirement instead of the uptime requirement for assigning the
   * Stable flag, 0 if not, or -1 if the authority doesn't report this
-   * information. */
+   * information.
+   *
+   * @since 1.1.0
+   */
  public int getEnoughMtbfInfo();

-  /* Return 1 if the authority has enough measured bandwidths that it'll
+  /**
+   * Return 1 if the authority has enough measured bandwidths that it'll
   * ignore the advertised bandwidth claims of routers without measured
   * bandwidth, 0 if not, or -1 if the authority doesn't report this
-   * information. */
+   * information.
+   *
+   * @since 1.1.0
+   */
  public int getIgnoringAdvertisedBws();

-  /* Return status entries, one for each contained bridge. */
+  /**
+   * Return status entries for each contained bridge, with map keys being
+   * SHA-1 digests of SHA-1 digest of the bridges' public identity keys,
+   * encoded as 40 upper-case hexadecimal characters.
+   *
+   * @since 1.0.0
+   */
  public SortedMap<String, NetworkStatusEntry> getStatusEntries();
 }

--- a/src/org/torproject/descriptor/BridgePoolAssignment.java
+++ b/src/org/torproject/descriptor/BridgePoolAssignment.java
-/* Copyright 2012--2015 The Tor Project
+/* Copyright 2012--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

 import java.util.SortedMap;

+/**
+ * Contains a sanitized list of bridges together with the distribution
+ * pools they have been assigned to by the bridge distribution service
+ * BridgeDB.
+ *
+ * <p>BridgeDB receives bridge network statuses
+ * ({@link BridgeNetworkStatus}) from the bridge authority, assigns these
+ * bridges to persistent distribution rings, and hands them out to bridge
+ * users.  BridgeDB periodically dumps the list of running bridges with
+ * information about the rings, subrings, and file buckets to which they
+ * are assigned to a local file.</p>
+ *
+ * <p>Details about sanitizing bridge pool assignments can be found
+ * <a href="https://collector.torproject.org/#type-bridge-pool-assignment">here</a>.
+ * </p>
+ *
+ * @since 1.0.0
+ */
 public interface BridgePoolAssignment extends Descriptor {

-  /* Return the publication time of this bridge pool assignment list. */
+  /**
+   * Return the time in milliseconds since the epoch when this descriptor
+   * was published.
+   *
+   * @since 1.0.0
+   */
  public long getPublishedMillis();

-  /* Return the entries contained in this bridge pool assignment list with
-   * map keys being bridge fingerprints and map values being assignment
-   * strings, e.g. "https ring=3 flag=stable". */
+  /**
+   * Return the entries contained in this bridge pool assignment list
+   * with map keys being SHA-1 digests of SHA-1 digest of the bridges'
+   * public identity keys, encoded as 40 upper-case hexadecimal
+   * characters, and map values being assignment strings, e.g.
+   * {@code "https ring=3 flag=stable"}.
+   *
+   * @since 1.0.0
+   */
  public SortedMap<String, String> getEntries();
 }

--- a/src/org/torproject/descriptor/BridgeServerDescriptor.java
+++ b/src/org/torproject/descriptor/BridgeServerDescriptor.java
-/* Copyright 2015 The Tor Project
+/* Copyright 2015--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

-/* Contains a bridge server descriptor. */
+/**
+ * Contains a sanitized bridge server descriptor.
+ *
+ * <p>Sanitized bridge server descriptors share many contents with relay
+ * server descriptors ({@link RelayServerDescriptor}), which is why they
+ * share a common superinterface ({@link ServerDescriptor}).  The main
+ * purpose of having two subinterfaces is being able to distinguish
+ * descriptor types more easily.</p>
+ *
+ * <p>Details about sanitizing bridge server descriptors can be found
+ * <a href="https://collector.torproject.org/#type-bridge-server-descriptor">here</a>.
+ * </p>
+ *
+ * @since 1.1.0
+ */
 public interface BridgeServerDescriptor extends ServerDescriptor {

 }

--- a/src/org/torproject/descriptor/Descriptor.java
+++ b/src/org/torproject/descriptor/Descriptor.java
-/* Copyright 2011--2015 The Tor Project
+/* Copyright 2011--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

 import java.util.List;

-/* Store meta-data about how a descriptor was downloaded or read from
- * disk. */
+/**
+ * Superinterface for any descriptor with access to generic information
+ * about the descriptor.
+ *
+ * @since 1.0.0
+ */
 public interface Descriptor {

-  /* Return the raw descriptor bytes. */
+  /**
+   * Return the raw descriptor bytes.
+   *
+   * @since 1.0.0
+   */
  public byte[] getRawDescriptorBytes();

-  /* Return the (possibly empty) list of annotations. */
+  /**
+   * Return the (possibly empty) list of annotations in the format
+   * {@code "@key( value)*"}.
+   *
+   * @since 1.0.0
+   */
  public List<String> getAnnotations();

-  /* Return any unrecognized lines when parsing this descriptor, or an
-   * empty list if there were no unrecognized lines. */
+  /**
+   * Return any unrecognized lines when parsing this descriptor, or an
+   * empty list if there were no unrecognized lines.
+   *
+   * @since 1.0.0
+   */
  public List<String> getUnrecognizedLines();
 }

--- a/src/org/torproject/descriptor/DescriptorCollector.java
+++ b/src/org/torproject/descriptor/DescriptorCollector.java
-/* Copyright 2015 The Tor Project
+/* Copyright 2015--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

 import java.io.File;

-/** Fetch descriptors from the CollecTor service available at
- * https://collector.torproject.org/ and store them to a local
- * directory. */
+/**
+ * Descriptor source that synchronizes descriptors from the CollecTor
+ * service to a given local directory.
+ *
+ * <p>This type is not a descriptor source in the proper sense, because it
+ * does not produce descriptors by itself.  But it often creates the
+ * prerequisites for reading descriptors from disk using
+ * {@link DescriptorReader}.</p>
+ *
+ * <p>Code sample:</p>
+ * <pre>{@code
+ * DescriptorCollector descriptorCollector =
+ *     DescriptorSourceFactory.createDescriptorCollector();
+ * descriptorCollector.collectDescriptors(
+ *     // Download from Tor's main CollecTor instance,
+ *     "https://collector.torproject.org",
+ *     // include network status consensuses and relay server descriptors
+ *     new String[] { "/recent/relay-descriptors/consensuses/",
+ *     "/recent/relay-descriptors/server-descriptors/" },
+ *     // regardless of last-modified time,
+ *     0L,
+ *     // write to the local directory called in/,
+ *     new File("in"),
+ *     // and delete extraneous files that do not exist remotely anymore.
+ *     true);
+ * }</pre>
+ *
+ * @since 1.0.0
+ */
 public interface DescriptorCollector {

  /**
@@ -15,17 +42,18 @@ public interface DescriptorCollector {
   * anymore.
   *
   * @param collecTorBaseUrl CollecTor base URL without trailing slash,
-   * e.g., "https://collector.torproject.org".
+   *     e.g., {@code "https://collector.torproject.org"}
   * @param remoteDirectories Remote directories to collect descriptors
-   * from, e.g., "/recent/relay-descriptors/server-descriptors/".  Only
-   * files in this directory will be collected, no files in its sub
-   * directories.
+   *     from, e.g.,
+   *     {@code "/recent/relay-descriptors/server-descriptors/"}, without
+   *     processing subdirectories unless they are explicitly listed
   * @param minLastModified Minimum last-modified time in milliseconds of
-   * files to be collected.  Set to 0 for collecting all files.
-   * @param localDirectory Directory where collected files will be
-   * written.
+   *     files to be collected, or 0 for collecting all files
+   * @param localDirectory Directory where collected files will be written
   * @param deleteExtraneousLocalFiles Whether to delete all local files
-   * that do not exist remotely anymore.
+   *     that do not exist remotely anymore
+   *
+   * @since 1.0.0
   */
  public void collectDescriptors(String collecTorBaseUrl,
      String[] remoteDirectories, long minLastModified,

--- a/src/org/torproject/descriptor/DescriptorDownloader.java
+++ b/src/org/torproject/descriptor/DescriptorDownloader.java
-/* Copyright 2011--2015 The Tor Project
+/* Copyright 2011--2016 The Tor Project
 * See LICENSE for licensing information */
+
 package org.torproject.descriptor;

 import java.util.Iterator;
 import java.util.Set;

-/* Download relay descriptors from directory mirrors or authorities. */
+/**
+ * Descriptor source that downloads relay descriptors from directory
+ * authorities or mirrors.
+ *
+ * <p>Downloading descriptors is done in a batch which starts after
+ * setting any configuration options and initiating the download
+ * process.</p>
+ *
+ * @since 1.0.0
+ */
 public interface DescriptorDownloader {

-  /* Add a directory authority to download descriptors from.  A directory
-   * authority is only required for downloading network status vote and
-   * will be used when no directory mirrors are available. */
+  /**
+   * Add a directory authority to download descriptors from, which is
+   * only required for downloading network status votes and will be used
+   * when no directory mirrors are available.
+   *
+   * @since 1.0.0
+   */
  public void addDirectoryAuthority(String nickname, String ip,
      int dirPort);

-  /* Add a directory mirror to download descriptors from.  Directory
-   * mirrors are preferred when downloading descriptors, except for
-   * network status votes which are only available on directory
-   * authorities. */
+  /**
+   * Add a directory mirror to download descriptors from, which is
+   * preferred for downloading descriptors, except for network status
+   * votes which are only available on directory authorities.
+   *
+   * @since 1.0.0
+   */
  public void addDirectoryMirror(String nickname, String ip, int dirPort);

-  /* Include the current network status consensus in the downloads. */
+  /**
+