GitLab is used only for code review, issue tracking and project management. Canonical locations for source code are still https://gitweb.torproject.org/ https://git.torproject.org/ and git-rw.torproject.org.

CHANGELOG.rst 15.8 KB
Newer Older
juga  's avatar
juga committed
1 2 3 4 5 6 7 8 9
Changelog
=========

All notable changes to this project will be documented in this file.

The format is based on `Keep a
Changelog <http://keepachangelog.com/en/1.0.0/>`__ and this project
adheres to `Semantic Versioning <http://semver.org/spec/v2.0.0.html>`__.

juga  's avatar
juga committed
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53
v1.1.0 (2019-03-27)
-------------------

New
~~~

- V3bwfile: Report excluded relays.
  Closes: #28565.
- V3bwfile: Add time to report half network.
  Closes: #28983
- Destination: Recover destination when it failed.
  Closes: #29589.
- V3bwfile: Report relays that fail to be measured.
  Closes: #28567.
- V3bwfile: Report relays that are not measured measured.
  Closes: #28566
- V3bwfile: Add KeyValues to monitor relays.
  Closes: #29591.
- Docs: document that authorities are not measured.
  Closes: #29722
- Scanner: Warn when there is no progress.
  Closes: #28652

Fix
~~~
- v3bwfile: Report relays even when they don't reach a minimum number.
  Closes: #29853.
- Minor fixes. Closes #29891.
- Relaylist: Convert consensus bandwidth to bytes.


v1.0.5 (2019-03-06)
-------------------

- Release v1.0.5.
  this time with the correct version

v1.0.4 (2019-03-06)
-------------------

- Release v1.0.4.
  because there was a commit missing between `1.0.3` and `1.0.4-dev0`
  and what is released as `1.0.3` has version `1.0.4-dev0` and it
  can not be fixed now.
juga  's avatar
juga committed
54

juga  's avatar
juga committed
55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138
v1.0.3 (2019-02-28)
-------------------

Fixed
~~~~~~

- scanner: check that ResultDump queue is not full
  Fixes bug #28866. Bugfix v0.1.0.
- config: set stdout log level to cli argument. Closes: #29199
- cleanup: Use getpath to get configuration paths. Bugfix v0.7.0.
- destination: stop running twice usability tests.
  Fixes bug #28897. Bugfix v0.3.0
- globals, stem: explain where torrc options are.
  Fixes bug #28646. Bugfix v0.4.0
- stem: disable pad connections. Fixes bug 28692. Bugfix v0.4.0
- generate: Load all results, including error ones.
  Closes #29568. Bugfix v0.4.0 (line introduced in v0.1.0).
- relayprioritizer: Stop prioritizing relays that tend to fail.
  Fixes bug #28868. Bugfix v0.1.0
- circuitbuilder: Stop building the circuit 3 times.
  Fixes bug #29295. Bugfix v0.1.0.
- docs: add verify option to man and example.
  Closes bug #28788. Bugfix v0.4.0.
- CI: run scanner using the test network. Fixes bug #28933. Bugfix v0.1.0.
- scanner: catch SIGINT in the main loop. Fixes bug #28869. Bugfix v0.1.0.
- Stop including tests network as binary blob. Fixes bug #28590. Bugfix v0.4.0.
- relaylist: remove assertions that fail measurement.
  Closes #28870. Bugfix v0.4.0
- config: Use configuration provided as argument.
  Fixes bug #28724. Bugfix v0.7.0.
- stem: parse torrc options that are only a key.
  Fixes bug #28715. Bugfix v0.1.1
- stem: Stop merging multiple torrc options with the same name.
  Fixes bug #28738. Bugfix v0.1.1
- docs: add note about syslog when running systemd.
  Closes bug #28761. Bugfix v0.6.0
- CI: include deb.torproject.org key.
  Closes #28922. Bugfix v1.0.3-dev0
- config: stop allowing http servers without tls.
  Fixes bug #28789. Bugfix v0.2.0.
- Make info level logs more clear and consistent.
  Closes bug #28736. Bugfix v0.3.0.
- CI: check broken links in the docs. Closes #28670.
- docs: add scanner and destination requirements.
  Closes bug #28647. Bugfix v0.4.0
- generate: use round_digs variable name in methods.
  Closes bug #28602. Bugfix 1.0.3-dev0
- docs: Change old broken links in the documentation. Closes #28662.
- docs: replace http by https in links. Closes #28661.
- Fix git repository link. Fixes bug #28762. Bugfix v1.0.0.
- docs: add example destination in DEPLOY. Closes #28649.
- docs: Change links to be interpreted by ReST. Closes #28648.
- Force rtfd.io to install the package. Closes bug #28601.
- config: continue when the file is not found. Closes: #28550.
- Stop resolving domains locally and check same flags for the 2nd hop.
  Closes bug #28458, #28471. Bugfix 1.0.4.
- Limit the relays' bandwidth to their consensus bandwidth. Closes #28598.
- globals: add torrc logging options. Closes #28645. Bugfix v0.2.0.
- Limit bandwidth to the relay MaxAdvertisedBandwidth
  Fixes bug #28588. Bugfix 0.8.0.
- Exclude results, then check for the minimum number. Closes bug 28572.
- Make sbws round to 3 significant figures in torflow rounding mode.
  Bugfix on 27337 in sbws 1.0. Part of 28442.

Changed
~~~~~~~~

- tests: remove unused testnets. Fixes bug #29046. Bugfix v0.4.0.
- scanner, destination: Log all possible exceptions.
- docs: Update/improve documentation on how the scanner/generator work.
  Closes: #29149
- Requests: Change make_session to use the TimedSession.
- CI: change to Ubuntu Xenial.
- docs: stop editing changelog on every bug/ticket. Closes ticket #28572.
- Change sbws scaling method to torflow. Closes: #28446.
- Round bandwidths to 2 significant digits by default.
  Implements part of proposal 276. Implements 28451.

Added
~~~~~~

- Send scanner metadata as part of every HTTP request. Closes: #28741
- scanner: log backtrace when not progressing. Closes: 28932

juga  's avatar
juga committed
139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311
v1.0.2 (2018-11-10)
-------------------

Fixed
~~~~~

-  Update bandwidth file specification version in the ``generator``
   (#28366).
-  Use 5 "=" characters as terminator in the bandwidth files (#28379)

Changed
~~~~~~~

-  Include the headers about eligible relays in all the bandwidth files,
   not only in the ones that does not have enough eligible relays
   (#28365).

v1.0.1 (2018-11-01)
-------------------

Changed
~~~~~~~

-  Change default directories when sbws is run from a system service
   (#28268).

v1.0.0 (2018-10-29)
-------------------

**Important changes**:

-  ``generate`` includes extra statistics header lines when the number
   of eligible relays to include is less than the 60% of the network. It
   does not include the relays' lines.
-  Speed up ``scanner`` by disabling RTT measurements and waiting for
   measurement threads before prioritizing again the list of relays to
   measure.

Fixed
~~~~~

-  Update python minimal version in setup (#28043)
-  Catch unhandled exception when we fail to resolve a domain name
   (#28141)
-  Bandwidth filtered is the maximum between the bandwidth measurements
   and their mean, not the minimum (#28215)
-  Stop measuring the same relay by two threads(#28061)

Changed
~~~~~~~

-  Move ``examples/`` to ``docs/`` (#28040)
-  Number of results comparison and number of results away from each
   other are incorrect (#28041)
-  Stop removing results that are not away from some other X secs
   (#28103)
-  Use secs-away when provided instead of data\_period (#28105)
-  Disable measuring RTTs (#28159)
-  Rename bandwidth file keyvalues (#28197)

Added
-----

-  Write bw file only when the percentage of measured relays is bigger
   than 60% (#28062)
-  When the percentage of measured relays is less than the 60%, do not
   include the relays in the bandwidth file and instead include some
   statistics in the header (#28076)
-  When the percentage of measured relays is less than the 60% and it
   was more before, warn about it (#28155)
-  When the difference between the total consensus bandwidth and the
   total in the bandwidth lines is larger than 50%, warn (#28216)
-  Add documentation about how the bandwidth measurements are selected
   and scaled before writing them to the Bandwidth File (#27692)

v0.8.0 (2018-10-08)
-------------------

**Important changes**:

-  Implement Torflow scaling/aggregation to be able to substitute
   Torflow with sbws without affecting the bandwidth files results.
-  Change stem dependency to 1.7.0, which removes the need for
   ``dependency_links``
-  Update and cleanup documentation

Added
~~~~~

-  Add system physical requirements section to INSTALL (#26937)
-  Warn when there is not enough disk space (#26937)
-  Implement Torflow scaling (#27108)
-  Create methods to easy graph generation and obtain statistics to
   compare with current torflow results.(#27688)
-  Implement rounding bw in bandwidth files to 2 insignificant
   digits(#27337)
-  Filter results in order to include relays in the bandwidth file
   that:(#27338)
-  have at least two measured bandwidths
-  the measured bandwidths are within 24 hours of each other
-  have at least two descriptor observed bandwidths
-  the descriptor observed bandwidths are within 24 hours of each other

Fixed
~~~~~

-  Broken environment variable in default sbws config. To use envvar
   $FOO, write $$FOO in the config.
-  Stop using directory as argument in integration tests (#27342)
-  Fix typo getting configuration option to allow logging to file
   (#27960)
-  Set int type to new arguments that otherwise would be string (#27918)
-  Stop printing arguments default values, since they are printed by
   default (#27916)
-  Use dash instead of underscore in new cli argument names (#27917)

Changed
~~~~~~~

-  sbws install doc is confusing (#27341)
-  Include system and Python dependencies in ``INSTALL``.
-  Include dependencies for docs and tests in ``INSTALL``.
-  Point to ``DEPLOY`` to run sbws.
-  Remove obsolete sections in ``INSTALL``
-  Simplify ``DEPLOY``, reuse terms in the ``glossary``.
-  Remove obsolete ``sbws init`` from ``DEPLOY``.
-  Point to config documentation.
-  Add, unify and reuse terms in ``glossary``.
-  refactor v3bwfile (#27386): move scaling method inside class
-  use custom ``install_command`` to test installation commands while
   ``dependency_links`` is needed until #26914 is fixed. (#27704)
-  documentation cleanup (#27773)
-  split, merge, simplify, extend, reorganize sections and files
-  generate scales as Torflow by default (#27976)
-  Replace stem ``dependency_links`` by stem 1.7.0 (#27705). This also
   eliminates the need for custom ``install_command`` in tox.

v0.7.0 (2018-08-09)
-------------------

**Important changes**:

-  ``cleanup/stale_days`` is renamed to
   ``cleanup/data_files_compress_after_days``
-  ``cleanup/rotten_days`` is renamed to
   ``cleanup/data_files_delete_after_days``
-  sbws now takes as an argument the path to a config file (which
   contains ``sbws_home``) instead of ``sbws_home`` (which contains the
   path to a config file)

Added
~~~~~

-  Log line on start up with sbws version, platform info, and library
   versions (trac#26751)
-  Manual pages (#26926)

Fixed
~~~~~

-  Stop deleting the latest.v3bw symlink. Instead, do an atomic rename.
   (#26740)
-  State file for storing the last time ``sbws scanner`` was started,
   and able to be used for storing many other types of state in the
   future. (GH#166)
-  Log files weren't rotating. Now they are. (#26881)

Changed
~~~~~~~

-  Remove test data v3bw file and generate it from the same test.
   (#26736)
-  Stop using food terms for cleanup-related config options
juga  's avatar
juga committed
312
-  cleanup command now cleans up old v3bw files too (#26701)
juga  's avatar
juga committed
313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459
-  Make sbws more compatible with system packages: (#26862)
-  Allow a configuration file argument
-  Remove directory argument
-  Create minimal user configuration when running
-  Do not require to run a command to initialize
-  Initialize directories when running
-  Do not require configuration file inside directories specified by the
   configuration

v0.6.0 (2018-07-11)
-------------------

**Important changes**:

-  The way users configure logging has changed. No longer are most users
   expected to be familiar with how to configure python's standard
   logging library with a config file. Instead we've abstracted out the
   setting of log level, format, and destinations to make these settings
   more accessible to users. Expert users familiar with `the logging
   config file
   format <https://docs.python.org/3/library/logging.config.html#logging-config-fileformat>`__
   can still make tweaks.

Summary of changes:

-  Make logging configuration easier for the user.
-  Add UML diagrams to documentation. They can be found in
   docs/source/images/ and regenerated with ``make umlsvg`` in docs/.

Added
~~~~~

-  UML diagrams to documentation. In docs/ run ``make umlsvg`` to
   rebuild them. Requires graphviz to be installed.(GHPR#226)
-  Add metadata to setup.py, useful for source/binary distributions.
-  Add possibility to log to system log. (#26683)
-  Add option to cleanup v3bw files. (#26701)

Fixed
~~~~~

-  Measure relays that have both Exit and BadExit as non-exits, which is
   how clients would use them. (GH#217)
-  Could not init sbws because of a catch-22 related to logging
   configuration. Overhaul how logging is configured. (GH#186 GHPR#224)
-  Call write method of V3BWFile class from the object instance.
   (#26671)
-  Stop calculating median on empty list .(#26666)

Changed
~~~~~~~

-  Remove is\_controller\_ok. Instead catch possible controller
   exceptions and log them

Removed
~~~~~~~

-  Two parsing/plotting scripts in scripts/tools/ that can now be found
   at https://github.com/pastly/v3bw-tools

v0.5.0 (2018-06-26)
-------------------

**Important changes**:

-  Result format changed, causing a version bump to 4. Updating sbws to
   0.5.0 will cause it to ignore results with version less than 4.

Summary of changes:

-  Keep previously-generated v3bw files
-  Allow a relay to limit its weight based on
   RelayBandwidthRate/MaxAdvertisedBandwidth
-  1 CPU usage optimization
-  1 memory usage optimization

Added
~~~~~

-  Use a relay's {,Relay}BandwidthRate/MaxAdvertisedBandwidth as an
   upper bound on the measurements we make for it. (GH#155)
-  Ability to only consider results for a given relay valid if they came
   from when that relay is using its most recent known IP address.
   Thanks Juga. (GH#154 GHPR#199)
-  Maintenance script to help us find functions that are (probably) no
   longer being called.
-  Integration test(s) for RelayPrioritizer (GHPR#206)
-  Git/GitHub usage guidelines to CONTRIBUTING document (GH#208
   GHPR#215)

Fixed
~~~~~

-  Make relay priority calculations take only ~5% of the time they used
   to (3s vs 60s) by using sets instead of lists when selecting
   non-Authority relays. (GH#204)
-  Make relay list refreshing take much less time by not allowing worker
   threads to dogpile on the CPU. Before they would all start requesting
   descriptors from Tor at roughly the same time, causing us to overload
   our CPU core and make the process take unnecessarily long. Now we let
   one thread do the work so it can peg the CPU on its own and get the
   refresh done ASAP. (GH#205)
-  Catch a JSON decode exception on malformed results so sbws can
   continue gracefully (GH#210 GHPR#212)

Changed
~~~~~~~

-  Change the path where the Bandwidth List files are generated: now
   they are stored in ``v3bw`` directory, named ``YYmmdd_HHMMSS.v3bw``,
   and previously generated ones are kept. A ``latest.v3bw`` symlink is
   updated. (GH#179 GHPR#190)
-  Code refactoring in the v3bw classes and generation area
-  Replace v3bw-into-xy bash script with python script to handle a more
   complex v3bw file format (GH#182)

v0.4.1 (2018-06-14)
-------------------

Changed
~~~~~~~

-  If the relay to measure is an exit, put it in the exit position and
   choose a non-exit to help. Previously the relay to measure would
   always be the first hop. (GH#181)
-  Try harder to find a relay to help measure the target relay with two
   changes. Essentially: (1) Instead of only picking from relays that
   are 1.25 - 2.00 times faster than it by consensus weight, try (in
   order) to find a relay that is at least 2.00, 1.75, 1.50, 1.25, or
   v1.00 times as fast. If that fails, instead of giving up, (2) pick the
   fastest relay in the network instead of giving up. This compliments
   the previous change about measuring target exits in the exit
   position.

Fixed
~~~~~

-  Exception that causes sbws to fall back to one measurement thread. We
   first tried fixing something in this area with ``88fae60bc`` but
   neglected to remember that ``.join()`` wants only string arguments
   and can't handle a ``None``. So fix that.
-  Exception when failing to get a relay's ``ed25519_master_key`` from
   Tor and trying to do ``.rstrip()`` on a None.
-  ``earliest_bandwidth`` being the newest bw not the oldest (thanks
   juga0)
-  ``node_id`` was missing the character "$" at the beginning