Fix bugs/issues with consolidate_stats
Quoting karsten:
I think there's a bug in this script: Whenever we skip a line in a .data file, because that line represents a failure, we might get out of sync with the .extradata file and stop writing any data to .mergedata. You should be able to reproduce this bug with the Torperf 50KB run (https://metrics.torproject.org/data/torperf-50kb.data https://metrics.torproject.org/data/torperf-50kb.extradata). The last line in the result has CIRC_ID=4384. If I delete the line in .extradata starting with CIRC_ID=4397, the result has more entries than before. I think the fix is to distinguish between absolute slack of up to 1 second and a time difference of, say, more than 1 minute.
Also, is there a way to include timed out runs in the .mergedata, too? We do include failures, so by including timeouts, we wouldn't have to parse the original files for timeouts/failures anymore. This could be a new ticket, I'd just like to know whether it's possible.