Missing recent bridge descriptors cause trouble with history file on fresh Onionoo instance
It's been 3 days now that CollecTor provided the last sanitized bridge descriptors, and this means that its recent/
directory is now empty. Apparently, we don't handle this unusual situation very well:
Exception in thread "main" java.lang.IllegalStateException: Operation is not permitted before finishing to read.
at org.torproject.descriptor.impl.DescriptorReaderImpl.getExcludedFiles(DescriptorReaderImpl.java:68)
at org.torproject.onionoo.updater.DescriptorQueue.writeHistoryFile(DescriptorQueue.java:113)
at org.torproject.onionoo.updater.DescriptorSource.writeHistoryFiles(DescriptorSource.java:222)
at org.torproject.onionoo.cron.Main.shutDown(Main.java:195)
at org.torproject.onionoo.cron.Main.run(Main.java:130)
at org.torproject.onionoo.cron.Main.runOrScheduleExecutions(Main.java:102)
at org.torproject.onionoo.cron.Main.main(Main.java:34)
Working hotfix:
diff --git a/src/main/java/org/torproject/onionoo/updater/DescriptorQueue.java b/src/main/java/org/torproject/onionoo/updater/DescriptorQueue.java
index c19fabd..0f3e9e7 100644
--- a/src/main/java/org/torproject/onionoo/updater/DescriptorQueue.java
+++ b/src/main/java/org/torproject/onionoo/updater/DescriptorQueue.java
@@ -108,6 +108,11 @@ class DescriptorQueue {
if (this.historyFile == null) {
return;
}
+ if (null == this.descriptors) {
+ log.warn("Not writing history file {}, because we did not read a single "
+ + "descriptor from {}.", this.historyFile, this.directory);
+ return;
+ }
SortedMap<String, Long> excludedAndParsedFiles = new TreeMap<>();
excludedAndParsedFiles.putAll(
this.descriptorReader.getExcludedFiles());
Log output:
2018-07-09 19:41:28,344 DEBUG o.t.o.u.DescriptorSource:220 Writing parse histories for recent descriptors...
2018-07-09 19:41:28,347 WARN o.t.o.u.DescriptorQueue:112 Not writing history file status/collector.torproject.org/bridge-server-hi
story, because we did not read a single descriptor from status/collector.torproject.org/recent/bridge-descriptors/server-descripto
rs.
2018-07-09 19:41:28,347 WARN o.t.o.u.DescriptorQueue:112 Not writing history file status/collector.torproject.org/bridge-extrainfo
-history, because we did not read a single descriptor from status/collector.torproject.org/recent/bridge-descriptors/extra-infos.
2018-07-09 19:41:28,347 WARN o.t.o.u.DescriptorQueue:112 Not writing history file status/collector.torproject.org/bridge-status-hi
story, because we did not read a single descriptor from status/collector.torproject.org/recent/bridge-descriptors/statuses.
2018-07-09 19:41:28,347 INFO o.t.o.cron.Main:196 Wrote parse histories
Let's look more into this and possibly finding a better fix as long as the situation exists that we don't have recent bridge descriptors from CollecTor.
To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information