GitLab

Today I realized that metrics-web does not delete any extraneous local files since we made DescriptorIndexCollector the default in metrics-lib version 1.6.0.

Turns out that the following metrics-web patch works around the issue:

diff --git a/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java b/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
index 499dff9..39b3b69 100644
--- a/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
+++ b/modules/collectdescs/src/main/java/org/torproject/metrics/collectdescs/Main.java
@@ -17,14 +17,14 @@ public class Main {
         DescriptorSourceFactory.createDescriptorCollector();
     collector.collectDescriptors(
         "https://collector.torproject.org", new String[] {
-            "/recent/bridge-descriptors/extra-infos/",
-            "/recent/bridge-descriptors/server-descriptors/",
-            "/recent/bridge-descriptors/statuses/",
-            "/recent/exit-lists/",
-            "/recent/relay-descriptors/consensuses/",
-            "/recent/relay-descriptors/extra-infos/",
-            "/recent/relay-descriptors/server-descriptors/",
-            "/recent/torperf/"
+            "recent/bridge-descriptors/extra-infos/",
+            "recent/bridge-descriptors/server-descriptors/",
+            "recent/bridge-descriptors/statuses/",
+            "recent/exit-lists/",
+            "recent/relay-descriptors/consensuses/",
+            "recent/relay-descriptors/extra-infos/",
+            "recent/relay-descriptors/server-descriptors/",
+            "recent/torperf/"
         }, 0L, new File("../../shared/in"), true);
   }
 }

However, I think we need to fix this in metrics-lib by accepting remote paths with leading / for fetching remote files and deleting extraneous local files.

iwakeh, want to look into this?