GitLab

Before this change we would have replaced an existing "filters" field
with the filters applied in this run. But if the user runs `onionperf
filter` more than once to apply different filters, we'll want to list
all filters in the "filters" field.

This change also removes a couple of `self.`s, in particular
`self.analysis` and `self.filters`, because one Filtering object can
apply filters to more than one analysis.

This commit tweaks the recently added --drop-guards switch as follows:

 - Guards are dropped right at startup and then every N hours.
   Otherwise we might not receive the first round of GUARD NEW/UP
   events. It's unclear why we don't receive those events, but finding
   out might be time-consuming whereas dropping guards at startup is
   basically free.

 - Right after guards are dropped, circuit build timeouts are dropped,
   too, if supported by the Tor version. If the Tor version does not
   support this, there's going to be a warning, and the control log
   will simply not contain BUILDTIMEOUT_SET events.

Still part of the reopened #33399.

This new filter mode removes Tor circuits that don't match the
provided fingerprints and leaves TGen transfers/streams untouched. At
the same time the visualize mode only includes TGen transfers/streams
with an existing mapping between TGen transfers/streams and Tor
streams/circuits.

This patch changes the default behavior of the visualize mode. The
original behavior of visualizing TGen transfers/streams *without* an
existing mapping to Tor streams/circuits can be selected with the
--outer-join switch, even though that's rather an edge use case.

Another minor change is that the filtered analysis files is not
written with sort_keys=True anymore, which would have produced a newly
sorted file with keys in alphabetic order rather than the original
insert order. The result is an actually useful diff.

And clarify that we're leaving statistics unchanged as part of the
filtering.

 - Add a change log entry.
 - Pick are more sensible default for `drop_guards_interval_hours`,
   also to fix unit tests.

Implements #33260.

 - Change timeout back to 270 seconds and stallout back to 0 seconds.
 - Change initial pause to 300 seconds to keep default behavior
   unchanged.
 - Change model, so that pause_between starts in parallel to a stream,
   not when the stream is completed. This is the same behavior as
   before.

Also add a change log entry for all changes.

Thanks to Rob for pointing this out.

initial_pause        -> pause_initial
inter_transfer_pause -> pause_between

Thanks to Rob for the suggestion.

We tell TGen client to finish on its own by passing the count option to
the end node:
https://github.com/shadow/tgen/blob/master/doc/TGen-Options.md#end-options

This patch adds another argument to the function watchdog_thread_task(),
no_relaunch, which instructs the function to not re-launch its process
if it fails.

This patch removes the --oneshot subcommand and replaces it with several
new subcommands for OnionPerf's "measure" command:

    --tgen-start-pause          (Initial pause before file transfers.)
    --tgen-num-transfers        (Number of file transfers.)
    --tgen-intertransfer-pause  (Pause in between file transfers.)
    --tgen-transfer-size        (Size of each file transfer.)

By default, OnionPerf continues to run in "continuous" mode.  One can
simulate oneshot mode by running onionperf with the following flags:

    onionperf measure --tgen-num-transfers=1

In addition to the above subcommands, this patch improves the code base
by 1) adding a TGenConf class to hold TGen's configuration and by 2)
adding a TGenModelConf class to hold TGen's traffic model.

This fixes #33432.

Still part of #34218.

Latest TGenTools contain a fix for correctly computing the start time
of failed streams. We have to update our unit tests.

Related to #30362.

Fixes #40006.