CHANGELOG.md 9.26 KB
Newer Older
1
2
3
4
5
6
7
8
9
# Changelog

All notable changes to this project will be documented in this file.

The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)
and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0.html).

## [Unreleased]

10
11
12
13
### Fixed

- Update python minimal version in setup (#28043)

14
15
16
### Changed

- Move ``examples/`` to ``docs/`` (#28040)
17
18
- Number of results comparison and number of results away from each other are
  incorrect (#28041)
19
- Stop removing results that are not away from some other X secs (#28103)
20

juga's avatar
juga committed
21
22
23
24
25
26
27
28
29
## [0.8.0] - 2018-10-08

**Important changes**:

- Implement Torflow scaling/aggregation to be able to substitute Torflow with
  sbws without affecting the bandwidth files results.
- Change stem dependency to 1.7.0, which removes the need for `dependency_links``
- Update and cleanup documentation

30
31
### Added

32
- Add system physical requirements section to INSTALL (#26937)
33
- Warn when there is not enough disk space (#26937)
34
- Implement Torflow scaling (#27108)
35
36
- Create methods to easy graph generation and obtain statistics
  to compare with current torflow results.(#27688)
juga's avatar
juga committed
37
- Implement rounding bw in bandwidth files to 2 insignificant digits(#27337)
38
39
40
41
42
- Filter results in order to include relays in the bandwidth file that:(#27338)
  - have at least two measured bandwidths
  - the measured bandwidths are within 24 hours of each other
  - have at least two descriptor observed bandwidths
  - the descriptor observed bandwidths are within 24 hours of each other
43

44
45
46
47
### Fixed

- Broken environment variable in default sbws config. To use envvar $FOO, write
  $$FOO in the config.
48
- Stop using directory as argument in integration tests (#27342)
juga's avatar
juga committed
49
- Fix typo getting configuration option to allow logging to file (#27960)
50
- Set int type to new arguments that otherwise would be string (#27918)
51
52
- Stop printing arguments default values, since they are printed by default
  (#27916)
53
- Use dash instead of underscore in new cli argument names (#27917)
54

juga's avatar
juga committed
55
56
57
58
59
60
61
62
63
64
65
### Changed

- sbws install doc is confusing (#27341)
  - Include system and Python dependencies in `INSTALL`.
  - Include dependencies for docs and tests in `INSTALL`.
  - Point to `DEPLOY` to run sbws.
  - Remove obsolete sections in `INSTALL`
  - Simplify `DEPLOY`, reuse terms in the `glossary`.
  - Remove obsolete ``sbws init`` from `DEPLOY`.
  - Point to config documentation.
  - Add, unify and reuse terms in `glossary`.
juga's avatar
juga committed
66
- refactor v3bwfile (#27386): move scaling method inside class
67
68
- use custom ``install_command`` to test installation commands while
  ``dependency_links`` is needed until #26914 is fixed. (#27704)
69
70
- documentation cleanup (#27773)
  - split, merge, simplify, extend, reorganize sections and files
71
- generate scales as Torflow by default (#27976)
72
73
- Replace stem ``dependency_links`` by stem 1.7.0 (#27705). This also eliminates
  the need for custom ``install_command`` in tox.
juga's avatar
juga committed
74

Matt Traudt's avatar
Matt Traudt committed
75
76
## [0.7.0] - 2018-08-09

77
78
79
80
**Important changes**:

- `cleanup/stale_days` is renamed to `cleanup/data_files_compress_after_days`
- `cleanup/rotten_days` is renamed to `cleanup/data_files_delete_after_days`
Matt Traudt's avatar
Matt Traudt committed
81
82
83
- sbws now takes as an argument the path to a config file (which contains
  `sbws_home`) instead of `sbws_home` (which contains the path to a config
file)
84

85
86
87
88
### Added

- Log line on start up with sbws version, platform info, and library versions
(trac#26751)
juga's avatar
juga committed
89
- Manual pages (#26926)
90

91
92
93
94
### Fixed

- Stop deleting the latest.v3bw symlink. Instead, do an atomic rename.
  (#26740)
95
96
- State file for storing the last time `sbws scanner` was started, and able to
  be used for storing many other types of state in the future. (GH#166)
97
- Log files weren't rotating. Now they are. (#26881)
98

99
100
### Changed

101
- Remove test data v3bw file and generate it from the same test. (#26736)
102
103
- Stop using food terms for cleanup-related config options
- Cleanup command now cleans up old v3bw files too (#26701)
104
105
106
107
108
109
110
111
- Make sbws more compatible with system packages: (#26862)
  - Allow a configuration file argument
  - Remove directory argument
  - Create minimal user configuration when running
  - Do not require to run a command to initialize
  - Initialize directories when running
  - Do not require configuration file inside directories specified by the
    configuration
Matt Traudt's avatar
Matt Traudt committed
112

Matt Traudt's avatar
Matt Traudt committed
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
## [0.6.0] - 2018-07-11

**Important changes**:

- The way users configure logging has changed. No longer are most users
  expected to be familiar with how to configure python's standard logging
library with a config file. Instead we've abstracted out the setting of log
level, format, and destinations to make these settings more accessible to
users. Expert users familiar with [the logging config file format][logconffmt]
can still make tweaks.

Summary of changes:

- Make logging configuration easier for the user.
- Add UML diagrams to documentation. They can be found in docs/source/images/
  and regenerated with `make umlsvg` in docs/.

[logconffmt]: https://docs.python.org/3/library/logging.config.html#logging-config-fileformat

132
133
134
135
### Added

- UML diagrams to documentation. In docs/ run `make umlsvg` to rebuild them.
  Requires graphviz to be installed.(GHPR#226)
juga's avatar
juga committed
136
- Add metadata to setup.py, useful for source/binary distributions.
juga's avatar
juga committed
137
- Add possibility to log to system log. (#26683)
138
- Add option to cleanup v3bw files. (#26701)
139

140
141
142
143
### Fixed

- Measure relays that have both Exit and BadExit as non-exits, which is how
  clients would use them. (GH#217)
144
145
- Could not init sbws because of a catch-22 related to logging configuration.
  Overhaul how logging is configured. (GH#186 GHPR#224)
146
- Call write method of V3BWFile class from the object instance. (#26671)
147
- Stop calculating median on empty list .(#26666)
Matt Traudt's avatar
Matt Traudt committed
148

149
150
### Changed

juga's avatar
juga committed
151
- Remove is_controller_ok. Instead catch possible controller exceptions and
152
153
log them

154
155
156
157
### Removed

- Two parsing/plotting scripts in scripts/tools/ that can now be found at
<https://github.com/pastly/v3bw-tools>
Matt Traudt's avatar
Matt Traudt committed
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173

## [0.5.0] - 2018-06-26

**Important changes**:

- Result format changed, causing a version bump to 4. Updating sbws to 0.5.0
  will cause it to ignore results with version less than 4.

Summary of changes:

- Keep previously-generated v3bw files
- Allow a relay to limit its weight based on
  RelayBandwidthRate/MaxAdvertisedBandwidth
- 1 CPU usage optimization
- 1 memory usage optimization

174
175
176
177
### Added

- Use a relay's {,Relay}BandwidthRate/MaxAdvertisedBandwidth as an upper bound
  on the measurements we make for it. (GH#155)
Matt Traudt's avatar
Matt Traudt committed
178
179
180
- Ability to only consider results for a given relay valid if they came from
  when that relay is using its most recent known IP address. Thanks Juga.
(GH#154 GHPR#199)
Matt Traudt's avatar
Matt Traudt committed
181
182
- Maintenance script to help us find functions that are (probably) no longer
  being called.
183
- Integration test(s) for RelayPrioritizer (GHPR#206)
184
- Git/GitHub usage guidelines to CONTRIBUTING document (GH#208 GHPR#215)
Matt Traudt's avatar
Matt Traudt committed
185

186
187
188
189
190
### Fixed

- Make relay priority calculations take only ~5% of the time they used to (3s
  vs 60s) by using sets instead of lists when selecting non-Authority relays.
(GH#204)
191
192
193
194
195
196
- Make relay list refreshing take much less time by not allowing worker threads
  to dogpile on the CPU. Before they would all start requesting descriptors
from Tor at roughly the same time, causing us to overload our CPU core and make
the process take unnecessarily long. Now we let one thread do the work so it
can peg the CPU on its own and get the refresh done ASAP.
(GH#205)
197
198
- Catch a JSON decode exception on malformed results so sbws can continue
  gracefully (GH#210 GHPR#212)
199

Matt Traudt's avatar
Matt Traudt committed
200
### Changed
201
202
203
204

- Change the path where the Bandwidth List files are generated: now they are
  stored in `v3bw` directory, named `YYmmdd_HHMMSS.v3bw`, and previously
generated ones are kept. A `latest.v3bw` symlink is updated. (GH#179 GHPR#190)
205
- Code refactoring in the v3bw classes and generation area
206
207
- Replace v3bw-into-xy bash script with python script to handle a more complex
  v3bw file format (GH#182)
208

Matt Traudt's avatar
Matt Traudt committed
209
## [0.4.1] - 2018-06-14
Matt Traudt's avatar
Matt Traudt committed
210

211
212
213
214
215
216
217
218
219
220
221
222
223
### Changed

- If the relay to measure is an exit, put it in the exit position and choose a
  non-exit to help. Previously the relay to measure would always be the first
hop. (GH#181)
- Try harder to find a relay to help measure the target relay with two changes.
  Essentially: (1) Instead of only picking from relays that are 1.25 - 2.00
times faster than it by consensus weight, try (in order) to find a relay that
is at least 2.00, 1.75, 1.50, 1.25, or 1.00 times as fast. If that fails,
instead of giving up, (2) pick the fastest relay in the network instead of
giving up. This compliments the previous change about measuring target exits in
the exit position.

224
225
226
227
228
229
### Fixed

- Exception that causes sbws to fall back to one measurement thread. We first
  tried fixing something in this area with `88fae60bc` but neglected to
remember that `.join()` wants only string arguments and can't handle a `None`.
So fix that.
230
231
- Exception when failing to get a relay's `ed25519_master_key` from Tor and
  trying to do `.rstrip()` on a None.
232
- `earliest_bandwidth` being the newest bw not the oldest (thanks juga0)
juga's avatar
juga committed
233
- `node_id` was missing the character "$" at the beginning
234

Matt Traudt's avatar
Matt Traudt committed
235
236
[Unreleased]: https://github.com/pastly/simple-bw-scanner/compare/v0.7.0...master
[0.7.0]: https://github.com/pastly/simple-bw-scanner/compare/v0.6.0...v0.7.0
Matt Traudt's avatar
Matt Traudt committed
237
[0.6.0]: https://github.com/pastly/simple-bw-scanner/compare/v0.5.0...v0.6.0
Matt Traudt's avatar
Matt Traudt committed
238
[0.5.0]: https://github.com/pastly/simple-bw-scanner/compare/v0.4.1...v0.5.0
Matt Traudt's avatar
Matt Traudt committed
239
[0.4.1]: https://github.com/pastly/simple-bw-scanner/compare/v0.4.0...v0.4.1