Skip to content

arti-bench: add support for multiple samples & averaging

eta requested to merge eta/arti:eta/292-1 into main

We now do multiple samples (configurable; default 3) per type of arti-bench benchmark run, and take a mean and median average of all data collected, in order to hopefully be a bit more resilient to random outliers / variation.

This uses some futures::stream::Stream hacks, which might result in more connections being made than required (and might impact the TTFB metrics somewhat, at least for downloading).

Results now get collected into a BenchmarkResults struct per type of benchmark, which will be in turn placed into a BenchmarkSummary in a later commit; this will also add the ability to serialize the latter struct out to disk, for future reference.

part of #292 (closed)

Merge request reports