1. 29 May, 2018 6 commits
    • Karsten Loesing's avatar
      Prepare for 1.1.0 release. · 1284c8ad
      Karsten Loesing authored
      1284c8ad
    • Karsten Loesing's avatar
      Replace Gson with Jackson. · 205d6da1
      Karsten Loesing authored
      Note that a side-effect of this change is that the generated news.json
      uses slightly different formatting. These are just cosmetic changes
      that don't have any effect on provided website content.
      
      Implements #26169.
      205d6da1
    • Karsten Loesing's avatar
      Be clearer about Onionoo's exit_addresses field. · 18280f6b
      Karsten Loesing authored
      Fixes #26031.
      18280f6b
    • Karsten Loesing's avatar
      Append _p to all plot_* and write_* parameters. · 7366398f
      Karsten Loesing authored
      This change makes our R code a little more readable in cases where
      parameter names match data column names.
      7366398f
    • Karsten Loesing's avatar
      Make all parameters in write_* functions optional. · 167e72b5
      Karsten Loesing authored
      We now permit parameters in write_* functions to be omitted. The
      effect is that we're not filtering if a parameter is missing, thus
      producing a CSV file with more rows.
      
      At the same time we're adding columns for data that was previously
      pre-determined by parameter values. For example, if a user specified a
      given country in a parameter, we didn't have to include a country
      column containing only that country. Now we need to put that column
      back.
      
      Implements #25383.
      167e72b5
    • Karsten Loesing's avatar
      Avoid underscores in write_* function parameters. · 3f1cafdb
      Karsten Loesing authored
      We added underscores to parameters in write_* functions when they
      would otherwise conflict with columns in the processed data. For
      example, if a graph supports a `country` parameter and the data also
      contains a `country` column, dplyr/tidyr won't know which `country` we
      mean. That's why we renamed the parameter to `country_`.
      
      However, we're soon going to make parameters optional, and if R
      receives a couple of parameters of which one has the name `country`,
      it can't match that to its `country_` parameter. We need to change the
      parameter back to `country` for this to work, which conflicts with the
      issue we were fixing earlier.
      
      Turns out there's a way to use the same name for parameter and data
      column: whenever we want to use the parameter, we use the
      quasiquotation operator `!!` which evaluates its argument early and
      inlines the result; and whenever we want to refer to the data column,
      we just refer to it by name, without that operator.
      
      Prepares #25383.
      3f1cafdb
  2. 26 May, 2018 2 commits
  3. 25 May, 2018 2 commits
    • Karsten Loesing's avatar
      Fix flaw in noise-removing code of hidserv module. · 52b1051f
      Karsten Loesing authored
      In the technical report that the hidserv module is based on we write:
      
      "When processing hidden-service statistics, we need to handle the fact
      that they have been obfuscated by relays. As first step, we're
      attempting to remove the additive Laplace-distributed noise by
      rounding up or down to the nearest multiple of bin_size. The idea is
      that it's most likely that noise was added to the closest right side
      of a bin than to the right side of another bin. In step two, we're
      subtracting half of bin_size, because the relay added between 0 and
      bin_size − 1 to the originally observed value."
      
      However, our code has a flaw: we use integer truncation rather which
      always rounds toward zero, whereas we really want to use the floor
      function which rounds towards negative infinity.
      
      The fix is to use Math.floorDiv() for the division rather than common
      integer division and truncation.
      
      Fixes #26022.
      52b1051f
    • Karsten Loesing's avatar
      Don't terminate the hidserv module when parsing descriptors. · 4333d066
      Karsten Loesing authored
      This issue came up when reprocessing descriptor archives for #26022:
      we'd terminate the hidserv module when running into an issue with
      parsing descriptors, where a possible issue is a consensus without a
      "bandwidth-weights" line.
      
      This seems wrong. It's okay to print out a warning in this case. But
      in general, we don't want to abort the entire module in this case.
      
      Related to #26022.
      4333d066
  4. 24 May, 2018 1 commit
    • Karsten Loesing's avatar
      Remove inconsistency between bandwidth history graphs. · 0eacbbec
      Karsten Loesing authored
      The number of bytes in all bandwidth histories differed from the sum
      of those numbers by Exit/Guard flag. The difference came from relays
      that reported bandwidth histories but that the directory authorities
      did not list as running.
      
      With this commit we stop producing bandwidth numbers that are not
      broken down by Exit/Guard flag, and we sum up those columns for the
      bandwidth graph.
      
      Implements #26015.
      0eacbbec
  5. 22 May, 2018 1 commit
  6. 18 May, 2018 2 commits
  7. 16 May, 2018 3 commits
    • Karsten Loesing's avatar
      Simplify censorship detector. · 27fe3f3b
      Karsten Loesing authored
      As part of the censorship detector, we're currently fitting a normal
      distribution to a set of approximately 50 values.
      
      After talking to George Danezis, the original author of our censorship
      detector, it's safe to approximate this step by computing the mean and
      standard deviation of these values and using those to parameterize the
      normal distribution.
      
      A test run with today's numbers produced the exact same result.
      
      The main benefit of this simplification is that it will be easier for
      others to reproduce our numbers, even without a statistics library as
      powerful as SciPy.
      27fe3f3b
    • Karsten Loesing's avatar
      Update to latest rs (Relay Search). · 168f5851
      Karsten Loesing authored
      168f5851
    • Karsten Loesing's avatar
  8. 15 May, 2018 1 commit
  9. 12 May, 2018 1 commit
  10. 11 May, 2018 1 commit
  11. 09 May, 2018 1 commit
  12. 07 May, 2018 3 commits
  13. 26 Apr, 2018 1 commit
  14. 23 Apr, 2018 4 commits
  15. 17 Apr, 2018 1 commit
  16. 16 Apr, 2018 1 commit
  17. 15 Apr, 2018 1 commit
  18. 06 Apr, 2018 1 commit
  19. 04 Apr, 2018 1 commit
  20. 03 Apr, 2018 2 commits
  21. 28 Mar, 2018 4 commits