CHANGELOG.md 17.1 KB
Newer Older
Hiro's avatar
Hiro committed
1
2
3
4
# Changes in version 2.xx.y - 2021-xx-yy

 * Medium changes

Hiro's avatar
Hiro committed
5
6
7
8
9
10
11
12
# Changes in version 2.19.0 - 2021-08-17

  * Medium changes
    - Expose overload-general fields in ServerDescriptor.

  * Minor changes
    - Document ExtraInfoDescriptor classes
    - Refactor javadoc to remove some style warnings
Iain Learmonth's avatar
Iain Learmonth committed
13
14


Iain Learmonth's avatar
Iain Learmonth committed
15
# Changes in version 2.18.0 - 2021-07-30
Iain Learmonth's avatar
Iain Learmonth committed
16
17

 * Medium changes
Iain Learmonth's avatar
Iain Learmonth committed
18
19
20
21
22
   - Expose overload-ratelimits and overload-fd-exhausted
     fields in ExtraInfoDescriptor.
   - OnionPerfAnalysisConverter discards circuits marked as
     filtered.
   - Adds BridgestrapStats to parse metrics from bridgestrap.
Iain Learmonth's avatar
Iain Learmonth committed
23
24


Iain Learmonth's avatar
Iain Learmonth committed
25
26
27
28
29
30
31
32
# Changes in version 2.17.0 - 2021-06-14

 * Medium changes
   - Detect GeoIP files automatically by filename.

 * Minor changes
   - Update to Apache Commons Logging 1.1.3.

Iain Learmonth's avatar
Iain Learmonth committed
33

Iain Learmonth's avatar
Iain Learmonth committed
34
# Changes in version 2.16.0 - 2021-05-02
35
36
37

 * Medium changes
   - Parse new NAT-based Snowflake lines.
Iain Learmonth's avatar
Iain Learmonth committed
38
   - Added a new parser for the IP Fire GeoIP database files used in core Tor.
Iain Learmonth's avatar
Iain Learmonth committed
39
   - Tests now depend on objenesis 2.2 (previously 2.1).
Iain Learmonth's avatar
Iain Learmonth committed
40
41
42
 * Minor changes
   - A lib folder is now included in the source distribution and does not need
     to be created prior to using the ant resolve task.
Karsten Loesing's avatar
Karsten Loesing committed
43
44


Karsten Loesing's avatar
Karsten Loesing committed
45
# Changes in version 2.15.0 - 2020-12-11
46
47
48
49

 * Medium changes
   - Parse version 3 onion service statistics contained in extra-info
     descriptors.
50
51
   - Optimize parsing of large files containing many descriptors.

52
53
54
 * Minor changes
   - Provide microdescriptor SHA-256 digest in hexadecimal encoding.

Karsten Loesing's avatar
Karsten Loesing committed
55

Karsten Loesing's avatar
Karsten Loesing committed
56
# Changes in version 2.14.0 - 2020-08-07
Karsten Loesing's avatar
Karsten Loesing committed
57

58
59
 * Medium changes
   - Extend Torperf results to provide error codes.
60
   - Parse OnionPerf analysis results format version 3.0.
61
62
   - Parse new ipv6-{write,read}-history and ipv6-conn-bi-direct
     lines in extra-info descriptors.
63

Karsten Loesing's avatar
Karsten Loesing committed
64

Karsten Loesing's avatar
Karsten Loesing committed
65
# Changes in version 2.13.0 - 2020-05-16
66
67

 * Medium changes
Karsten Loesing's avatar
Karsten Loesing committed
68
   - Parse OnionPerf analysis results format version 2.0.
69
70
   - Extend Torperf results to provide partial download times for 10,
     20, 50, 100, 200, and 500 KiB as well as 1, 2, and 5 MiB.
Karsten Loesing's avatar
Karsten Loesing committed
71

72
73
74
75
 * Minor changes
   - Include previously unknown error codes in Torperf results
     converted from OnionPerf analysis files.

Karsten Loesing's avatar
Karsten Loesing committed
76

Karsten Loesing's avatar
Karsten Loesing committed
77
78
79
80
81
# Changes in version 2.12.2 - 2020-04-30

 * Minor changes
   - Change the order of detecting descriptor types by content and/
     file name.
Karsten Loesing's avatar
Karsten Loesing committed
82
83


Karsten Loesing's avatar
Karsten Loesing committed
84
# Changes in version 2.12.1 - 2020-04-30
Karsten Loesing's avatar
Karsten Loesing committed
85

86
87
88
89
 * Minor changes
   - Change back how we treat xz-compressed files by leaving
     decompression to descriptor implementations.

Karsten Loesing's avatar
Karsten Loesing committed
90

Karsten Loesing's avatar
Karsten Loesing committed
91
# Changes in version 2.12.0 - 2020-04-30
Karsten Loesing's avatar
Karsten Loesing committed
92

93
94
95
96
 * Medium changes
   - Add parsing support for OnionPerf analysis files by converting
     and returning contained transfers as Torperf results.

Karsten Loesing's avatar
Karsten Loesing committed
97

Karsten Loesing's avatar
Karsten Loesing committed
98
# Changes in version 2.11.0 - 2020-04-13
Karsten Loesing's avatar
Karsten Loesing committed
99
100
101
102
103
104

 * Medium changes
   - Compute bandwidth file digests.
   - Parse bandwidth file header and bandwidth file digest in votes.
   - Parse bridge distribution requests in bridge server descriptors.
   - Parse authority fingerprint in bridge network statuses.
Karsten Loesing's avatar
Karsten Loesing committed
105

106
107
 * Minor changes
   - Avoid invoking overridable methods from constructors.
108
   - Make all descriptor instances serializable.
109
   - Simplify logging configuration.
110

Karsten Loesing's avatar
Karsten Loesing committed
111

Karsten Loesing's avatar
Karsten Loesing committed
112
# Changes in version 2.10.0 - 2020-01-15
113
114
115

 * Medium changes
   - Parse three newly added lines in snowflake statistics files.
Karsten Loesing's avatar
Karsten Loesing committed
116

117
118
119
 * Minor changes
   - Fix a NullPointerException when parsing an invalid crypto block
     starting with "-----END " rather than "-----BEGIN ".
120
121
   - Properly parse an authority's hostname from the "dir-source" line
     in a v2 network status.
122

Karsten Loesing's avatar
Karsten Loesing committed
123

124
125
126
127
# Changes in version 2.9.1 - 2019-11-09

 * Minor changes
   - Do not fail when processing index.json files with unknown fields.
Karsten Loesing's avatar
Karsten Loesing committed
128
129


Karsten Loesing's avatar
Karsten Loesing committed
130
# Changes in version 2.9.0 - 2019-11-01
Karsten Loesing's avatar
Karsten Loesing committed
131

132
133
134
 * Medium changes
   - Make NetworkStatusEntryImpl#parseSLine thread-safe.

Karsten Loesing's avatar
Karsten Loesing committed
135

Karsten Loesing's avatar
Karsten Loesing committed
136
# Changes in version 2.8.0 - 2019-10-18
137
138
139
140
141
142
143

 * Medium changes
   - Extend DescriptorReader#readDescriptors to support .gz-compressed
     files.
   - Add new BridgedbMetrics descriptor type.


Karsten Loesing's avatar
Karsten Loesing committed
144
# Changes in version 2.7.0 - 2019-09-06
145
146
147
148
149
150
151
152
153
154
155

 * Medium changes
   - Use Ivy for resolving external dependencies rather than relying
     on files found in Debian stable packages. Requires installing Ivy
     (using `apt-get install ivy`, `brew install ivy`, or similar) and
     running `ant resolve` (or `ant -lib /usr/share/java resolve`).
     Retrieved files are then copied to the `lib/` directory, except
     for dependencies on other metrics libraries that still need to be
     copied to the `lib/` directory manually. Current dependency
     versions resolved by Ivy are the same as in Debian stretch with
     few exceptions.
156
   - Remove Cobertura from the build process.
157
   - Add new SnowflakeStats descriptor type.
158
159


Karsten Loesing's avatar
Karsten Loesing committed
160
# Changes in version 2.6.2 - 2019-05-29
161
162
163
164
165

 * Medium changes
   - Recognize bandwidth files with @type annotation.


Karsten Loesing's avatar
Karsten Loesing committed
166
# Changes in version 2.6.1 - 2019-05-03
167
168
169
170
171

 * Medium changes
   - Fix a bug in recognizing descriptors as bandwidth files.


Karsten Loesing's avatar
Karsten Loesing committed
172
# Changes in version 2.6.0 - 2019-04-29
173
174

 * Medium changes
Karsten Loesing's avatar
Karsten Loesing committed
175
   - Stop signing jar files.
176
177
178
   - Add new BandwidthFile descriptor for parsed bandwidth files.


Karsten Loesing's avatar
Karsten Loesing committed
179
# Changes in version 2.5.0 - 2018-09-25
180
181
182
183
184
185
186

 * Medium changes
   - Go back to using Apache Commons Codec as base64 and hexadecimal
     codec rather than using JAXB which won't be available anymore
     after upgrading from Java 8 to 9. Applications must provide
     Apache Commons Codec 1.10 as dependency.

187
188
189
 * Minor changes
   - Make DescriptorCollector resume previously aborted downloads.

190

Karsten Loesing's avatar
Karsten Loesing committed
191
# Changes in version 2.4.0 - 2018-05-23
Karsten Loesing's avatar
Karsten Loesing committed
192
193
194
195
196
197
198

 * Medium changes
   - Replace Gson with Jackson. Applications must provide Jackson
     2.8.6 or compatible as dependency and do not need to provide Gson
     as dependency anymore.


Karsten Loesing's avatar
Karsten Loesing committed
199
# Changes in version 2.3.0 - 2018-04-18
200

201
202
203
204
 * Medium changes
   - Replace ServerDescriptor#getHiddenServiceDirVersions with
     ServerDescriptor#isHiddenServiceDir, because Tor has never
     supported versions in the hidden-service-dir descriptor line.
205
206
207
   - Add support for reading web server logs contained in tarballs by
     providing file names of both the tarball and of contained files
     to WebServerAccessLog.
208
209

 * Minor changes
210
211
   - Override logLines() method from LogDescriptor in
     WebServerAccessLog.
212
213
   - Use 1-minute connect and read timeouts for fetching CollecTor's
     index.json.
214
215


216
# Changes in version 2.2.0 - 2018-02-26
217
218
219
220
221

 * Major changes
   - Add new descriptor type WebServerAccessLog to parse web server
     access logs.

222
223
224
 * Minor changes
   - Add log message reporting progress reading tarballs.

225

Karsten Loesing's avatar
Karsten Loesing committed
226
# Changes in version 2.1.1 - 2017-10-09
227

Karsten Loesing's avatar
Karsten Loesing committed
228
 * Minor changes
229
230
231
232
   - Add new optional "build_revision" field to index.json with the
     Git revision supplied by the calling software.


Karsten Loesing's avatar
Karsten Loesing committed
233
# Changes in version 2.1.0 - 2017-09-15
234
235
236
237
238

 * Major changes
   - Update dependencies to versions available in Debian stretch:
     Apache Commons Compress 1.13, Apache Commons Lang 3.5, Google
     Gson 2.4, JUnit 4.12, Logback 1.1.9, SLF4J 1.7.22, and XZ 1.6.
Karsten Loesing's avatar
Karsten Loesing committed
239
   - Update to Java 8.
240

241
242
243
 * Minor changes
   - Retain trailing newline in Torperf results.

244

Karsten Loesing's avatar
Karsten Loesing committed
245
# Changes in version 2.0.0 - 2017-06-28
246

Karsten Loesing's avatar
Karsten Loesing committed
247
248
249
 * Major changes
   - Always use UTF-8 as charset rather than using the platform's
     default charset.
Karsten Loesing's avatar
Karsten Loesing committed
250
   - Remove all code that was deprecated in 1.x versions.
Karsten Loesing's avatar
Karsten Loesing committed
251
252
253
   - Remove previously non-deprecated but internally unused
     parseDescriptors(byte[], String)) method that returned a List and
     threw DescriptorParseException.
Karsten Loesing's avatar
Karsten Loesing committed
254

255
256
257
 * Minor changes
   - Replace custom ImplementationNotAccessibleException thrown by
     DescriptorSourceFactory with generic RuntimeException.
iwakeh's avatar
iwakeh committed
258
   - Rename jar files and release tarball to start with "metrics-lib".
259
260


Karsten Loesing's avatar
Karsten Loesing committed
261
# Changes in version 1.9.0 - 2017-06-21
262

263
264
265
266
267
268
269
270
271
272
273
274
 * Major changes
   - Simplify DescriptorReader by returning Descriptor instances
     rather than DescriptorFile instances containing Descriptors,
     deprecate DescriptorFile, and add a File reference to Descriptor.
   - Introduce a new UnparseableDescriptor to be returned by
     DescriptorParser and DescriptorReader if a descriptor cannot be
     parsed, as opposed to throwing a DescriptorParseException or
     skipping the entire descriptor file, respectively.

 * Medium changes
   - Let DescriptorParser return an Iterable instead of a List, which
     prepares parsing large descriptor files descriptor by descriptor.
275
276
277
   - Add new method to retrieve the raw descriptor length, rather than
     forcing applications to request (a copy of) raw descriptor bytes
     only to determine the raw descriptor length.
278

279
280
281
282
283
284
 * Minor changes
   - Fix a bug where NetworkStatusEntry's getMicrodescriptorDigests()
     and getMicrodescriptorDigestsSha256Base64() return hex strings
     rather than base64 strings.


285
286
287
288
289
290
# Changes in version 1.8.2 - 2017-06-16

 * Medium changes
   - Fix a regression where any DescriptorParseException thrown while
     parsing a descriptor is encapsulated and rethrown as
     RuntimeException.
Karsten Loesing's avatar
Karsten Loesing committed
291
292


Karsten Loesing's avatar
Karsten Loesing committed
293
294
295
296
297
# Changes in version 1.8.1 - 2017-06-08

 * Medium changes
   - Fix a regression in parsing microdescriptors and version 1
     directories introduced in 1.8.0.
Karsten Loesing's avatar
Karsten Loesing committed
298
299


Karsten Loesing's avatar
Karsten Loesing committed
300
# Changes in version 1.8.0 - 2017-06-07
301

302
303
304
305
306
 * Medium changes
   - Store raw descriptor contents as offset and length into a
     referenced byte[], rather than copying contents into a separate
     byte[] per descriptor.

307
308
309
310
 * Minor changes
   - Turn keyword strings into enums and use the appropriate enum sets
     and maps to avoid repeating string literals and to use more speedy
     collection types.
311
   - Simplify and avoid repetition in parse helper methods.
312
313
   - Fix a bug where Microdescriptor's getDigestSha256Base64() returns
     a hex string rather than a base64 string.
314
   - Move descriptor digest computation to DescriptorImpl.
315
316
   - Fix a bug in digest computation by making sure that the
     descriptor string actually contains the end token.
317
318
319
320
   - Fix a bug where both RelayDirectoryImpl and all NetworkStatusImpl
     subclasses fail to get indexes right if parts of raw descriptor
     strings contain non-ASCII chars.  In practice, this only affects
     version 1 directories which were last archived in 2007.
321
322


Karsten Loesing's avatar
Karsten Loesing committed
323
# Changes in version 1.7.0 - 2017-05-17
324

Karsten Loesing's avatar
Karsten Loesing committed
325
 * Medium changes
326
327
   - Fix a bug where unrecognized lines in extra-info descriptors
     below crypto blocks were silently skipped.
328
   - Add support for six new key-value pairs added by OnionPerf.
329
330
   - Fix a bug where DescriptorIndexCollector would not delete
     extraneous local files if remote paths start with /.
331
   - Add previously missing method to obtain the digest of a vote.
Karsten Loesing's avatar
Karsten Loesing committed
332
   - Deprecate setFailUnrecognizedDescriptorLines() in
333
334
335
     DescriptorParser and DescriptorReader and refer to
     getUnrecognizedLines() in Descriptor if applications really need
     to fail descriptors containing unrecognized lines.
336
   - Parse "padding-counts" lines in extra-info descriptors.
337

338
339
340
 * Minor changes
   - Accept extra arguments in statistics-related extra-info
     descriptor lines, as permitted by dir-spec.txt.
341
   - Streamline digest method names.
342

343

Karsten Loesing's avatar
Karsten Loesing committed
344
# Changes in version 1.6.0 - 2017-02-17
345

346
347
348
349
 * Major changes
   - Deprecate DescriptorDownloader in favor of the much more widely
     used DescriptorCollector.

350
351
352
353
 * Medium changes
   - Add two methods for loading and saving a parse history file in
     the descriptor reader to avoid situations where applications fail
     after all descriptors are read but before they are all processed.
iwakeh's avatar
iwakeh committed
354
355
356
   - Unify the build process by adding git-submodule metrics-base in
     src/build and removing all centralized parts of the build
     process.
357
358
   - Avoid deleting extraneous local descriptor files when collecting
     descriptors from CollecTor.
359
360
361
362
   - Turn the descriptor reader thread into a daemon thread, so that
     the application can decide at any time to stop consuming
     descriptors without having to worry about the reader thread not
     being done.
363
364
365
   - Parse "proto" lines in server descriptors, "pr" lines in status
     entries, and "(recommended|required)-(client|relay)-protocols"
     lines in consensuses and votes.
366
   - Parse "shared-rand-.*" lines in consensuses and votes.
367
368
   - Deprecate DescriptorCollectorImpl now that
     DescriptorIndexCollector is the default.
369
370


Karsten Loesing's avatar
Karsten Loesing committed
371
# Changes in version 1.5.0 - 2016-10-19
372

373
374
375
376
377
378
 * Major changes
   - Make the DescriptorCollector implementation that uses CollecTor's
     index.json file to determine which descriptor files to fetch the
     new default.  Applications must provide gson-2.2.4.jar or higher
     as dependency.

379
380
381
382
 * Minor changes
   - Avoid running into an IOException and logging a warning for it.


Karsten Loesing's avatar
Karsten Loesing committed
383
# Changes in version 1.4.0 - 2016-08-31
384

385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
 * Major changes
   - Add the Simple Logging Facade for Java (slf4j) for logging
     support rather than printing warnings to stderr.  Applications
     must provide slf4j-api-1.7.7.jar or higher as dependency and can
     optionally provide a compatible logging framework of their choice
     (java.util.logging, logback, log4j).

 * Medium changes
   - Add an alpha version of a DescriptorCollector implementation that
     is not enabled by default and that uses CollecTor's index.json
     file to determine which descriptor files to fetch.  Applications
     can enable this implementation by providing gson-2.2.4.jar or
     higher as dependency and setting property descriptor.collector to
     org.torproject.descriptor.index.DescriptorIndexCollector.

400
 * Minor changes
401
402
   - Include resource files in src/*/resources/ in the release
     tarball.
403
   - Move executable, source, and javadoc jar to generated/dist/.
404
405


406
407
408
409
410
411
412
# Changes in version 1.3.1 - 2016-08-01

 * Medium changes
   - Adapt to CollecTor's new date format to make DescriptorCollector
     work again.


Karsten Loesing's avatar
Karsten Loesing committed
413
# Changes in version 1.3.0 - 2016-07-06
414
415
416

 * Medium changes
   - Parse "package" lines in consensuses and votes.
417
418
419
420
421
422
423
424
   - Support more than one "directory-signature" line in a vote, which
     may become relevant when authorities start signing votes using
     more than one algorithm.
   - Provide directory signatures in consensuses and votes in a list
     rather than a map to support multiple signatures made using the
     same identity key digest but different algorithms.
   - Be more lenient about digest lengths in directory signatures
     which may be longer or shorter than 20 bytes.
425
   - Parse "tunnelled-dir-server" lines in server descriptors.
426

427
428
429
430
 * Minor changes
   - Stop reporting "-----END .*-----" lines in v2 network statuses as
     unrecognized.

431

Karsten Loesing's avatar
Karsten Loesing committed
432
# Changes in version 1.2.0 - 2016-05-31
433
434
435
436

 * Medium changes
   - Include the hostname in directory source entries of consensuses
     and votes.
437
   - Also accept \r\n as newline in Torperf results files.
438
439
   - Make unrecognized keys of Torperf results available together with
     the corresponding values, rather than just the whole line.
440
441
   - In Torperf results, recognize all percentiles of expected bytes
     read for 0 <= x <= 100 rather than just x = { 10, 20, ..., 90 }.
442
443
   - Rename properties for overriding default descriptor source
     implementation classes.
444
   - Actually return the signing key digest in network status votes.
445
   - Parse crypto parts in network status votes.
446
447
   - Document all public parts in org.torproject.descriptor and add
     an Ant target to generate Javadocs.
448

449
450
451
 * Minor changes
   - Include a Torperf results line with more than one unrecognized
     key only once in the unrecognized lines.
452
453
   - Make "consensus-methods" line optional in network statuses votes,
     which would mean that only method 1 is supported.
454
455
   - Stop reporting "-----END .*-----" lines in directory key
     certificates as unrecognized.
Karsten Loesing's avatar
Karsten Loesing committed
456
   - Add code used for benchmarking.
457

458

Karsten Loesing's avatar
Karsten Loesing committed
459
# Changes in version 1.1.0 - 2015-12-28
460
461

 * Medium changes
462
463
464
465
466
467
   - Parse flag thresholds in bridge network statuses, and parse the
     "ignoring-advertised-bws" flag threshold in relay network status
     votes.
   - Support parsing of .xz-compressed tarballs using Apache Commons
     Compress and XZ for Java.  Applications only need to add XZ for
     Java as dependency if they want to parse .xz-compressed tarballs.
iwakeh's avatar
iwakeh committed
468
469
470
471
472
   - Introduce a new ExitList.Entry type for exit list entries instead
     of the ExitListEntry type which is now deprecated.  The main
     difference between the two is that ExitList.Entry can hold more
     than one exit address and scan time which were previously parsed
     as multiple ExitListEntry instances.
473
474
475
476
477
478
   - Introduce four new types to distinguish between relay and bridge
     descriptors: RelayServerDescriptor, RelayExtraInfoDescriptor,
     BridgeServerDescriptor, and BridgeExtraInfoDescriptor.  The
     existing types, ServerDescriptor and ExtraInfoDescriptor, are
     still usable and will not be deprecated, because applications may
     not care whether a relay or a bridge published a descriptor.
479
480
481
482
483
   - Support Ed25519 certificates, Ed25519 master keys, SHA-256
     digests, and Ed25519 signatures thereof in server descriptors and
     extra-info descriptors, and support Ed25519 master keys in votes.
   - Include RSA-1024 signatures of SHA-1 digests of extra-info
     descriptors, which were parsed and discarded before.
484
   - Support hidden-service statistics in extra-info descriptors.
485
486
   - Support onion-key and ntor-onion-key cross certificates in server
     descriptors.
487

488
489
490
491
 * Minor changes
   - Start using Java 7 features like the diamond operator and switch
     on String, and use StringBuilder correctly in many places.

492

Karsten Loesing's avatar
Karsten Loesing committed
493
# Changes in version 1.0.0 - 2015-12-05
Karsten Loesing's avatar
Karsten Loesing committed
494
495

 * Major changes
496
497
   - This is the initial release after four years of development.
     Happy 4th birthday!