Changes
Page history
add more context to benchmark, now that it's out of cache
authored
Mar 17, 2021
by
anarcat
Show whitespace changes
Inline
Side-by-side
howto/benchmark.md
View page @
670aaf32
...
@@ -5,6 +5,9 @@ TPA.
...
@@ -5,6 +5,9 @@ TPA.
# HTTP load testing
# HTTP load testing
Those procedures were quickly established to compare various caching
software as part of the
[
cache service
](
howto/cache
)
setup.
## Common procedure
## Common procedure
1.
punch a hole in the firewall to allow the test server to access
1.
punch a hole in the firewall to allow the test server to access
...
@@ -55,10 +58,15 @@ Classic commandline:
...
@@ -55,10 +58,15 @@ Classic commandline:
ab2 -n 1000 -c 100 -X cache01.torproject.org https://example.com/
ab2 -n 1000 -c 100 -X cache01.torproject.org https://example.com/
`-X`
also doesn't work with ATS,
hacked
`/etc/hosts`
.
`-X`
also doesn't work with ATS,
modify
`/etc/hosts`
instead
.
## bombardier
## bombardier
We tested bombardier as an alternative to
`go-wrk`
in previous
benchmarks. The goal of using
`go-wrk`
was that it supported HTTP/2
(while
`wrk`
didn't), but
`go-wrk`
had performance issues, so we went
with the next best (and similar) thing.
Unfortunately, the
[
bombardier package in Debian
](
https://tracker.debian.org/pkg/bombardier
)
is
*not*
the HTTP
Unfortunately, the
[
bombardier package in Debian
](
https://tracker.debian.org/pkg/bombardier
)
is
*not*
the HTTP
benchmarking tool but a commandline game. It's still possible to
benchmarking tool but a commandline game. It's still possible to
install it in Debian with:
install it in Debian with:
...
@@ -116,6 +124,9 @@ the advantage of being already packaged in Debian. Simple cheat sheet:
...
@@ -116,6 +124,9 @@ the advantage of being already packaged in Debian. Simple cheat sheet:
echo "10.0.0.0 target.example.com" >> /etc/hosts
echo "10.0.0.0 target.example.com" >> /etc/hosts
wrk --latency -c 100 --duration 2m https://target.example.com/
wrk --latency -c 100 --duration 2m https://target.example.com/
The main disadvantage is that it doesn't (seem to) support HTTP/2 or
similarly advanced protocols.
## Other tools
## Other tools
Siege has trouble going above ~100 concurrent clients because of its
Siege has trouble going above ~100 concurrent clients because of its
...
@@ -138,3 +149,9 @@ limited, here's a set of interesting alternatives:
...
@@ -138,3 +149,9 @@ limited, here's a set of interesting alternatives:
Note that the Proto(col) and Features columns are not exhaustive: a
Note that the Proto(col) and Features columns are not exhaustive: a
tool
*might*
support (say) HTTPS, HTTP/2, or HTTP/3 even if it doesn't
tool
*might*
support (say) HTTPS, HTTP/2, or HTTP/3 even if it doesn't
explicitly mention it, although it's unlikely.
explicitly mention it, although it's unlikely.
It should be noted that very few (if any) benchmarking tools seem to
support HTTP/3 (or even QUIC) at this point. Even HTTP/2 support is
spotty: for example, while bombardier supports HTTP/2, it only does so
with the slower
`net/http`
library at the time of writing (2021). It's
unclear how many (if any) other projects to support HTTP/2 as well.