chutney verify: report client, bridge, and HS performance

There is a need to load-test Tor relays when bringing them up, and to performance-test tor in general, whether relaying or hidden services.

See: Tor BSD Underperformance https://lists.torproject.org/pipermail/tor-dev/2015-January/008048.html Simulate high Tor load https://lists.torproject.org/pipermail/tor-relays/2015-January/006203.html

It would be nice to have a command for this, that would execute the same tests, and then report performance as a series of MB/s figures for clients->exits, bridge clients->exits, and clients->hidden services.

Nick, is this SponsorR, or SponsorS, or both?

added SponsorR in Legacy / Trac SponsorS in Legacy / Trac TorCoreTeam201507 in Legacy / Trac component::core tor/chutney in Legacy / Trac owner::teor in Legacy / Trac priority::medium in Legacy / Trac resolution::implemented in Legacy / Trac status::closed in Legacy / Trac tor-bridge in Legacy / Trac tor-client in Legacy / Trac tor-hs in Legacy / Trac tor-perf in Legacy / Trac type::enhancement in Legacy / Trac labels

Tagged with appropriate sponsors per nickm in legacy/trac#14174 (moved)

Trac:
Keywords: tor-perf tor-hs tor-client tor-bridge deleted, tor-perf SponsorR SponsorS tor-hs tor-client tor-bridge added

If we're going to test performance, we need to report:

• time to contact server TCP SYN (if available)
• time to first response first TCP ACK (if available)
• time to connect second TCP ACK (if available)
• ongoing latency
• ongoing throughput

As connection time, latency, and throughput all impact perceived performance.

Alternately, we could leave this feature for Chutney 2.0, and just report performance each time we run chutney verify. This would work well with the configurable data volumes in legacy/trac#14174 (moved).

Trac:
Status: new to assigned
Summary: chutney perf: a new command that reports client, bridge, and HS performance to chutney verify: report client, bridge, and HS performance
Owner: nickm to teor

I have bundled up all these changes into an early draft in branch: bug14175-bandwidth on: https://github.com/teor2345/chutney.git

It may need to be split into separate commits, but there are some dependencies I have to work out first.

Note: The bandwidth parameters can be changed by modifying constants in TorNet.py. If we decide it's a good idea, I can do command-line arguments for these.

You'll likely need to change DATALEN in TorNet.py to 50MB or 100MB. (I just hung python or tor on EC2 using 1GB.)

As a sanity check, the script doesn't calculate bandwidth until it takes > 1 second, and has sent > 5MB.

Expect to see something like this:

Verifying data transmission:
Connecting:
  Exit to 127.0.0.1:4747 via client localhost:9008
  Exit to 127.0.0.1:4747 via client localhost:9009
Transmitting Data:
........................................................................................................................
Single Stream Bandwidth: 2034.50 Mbps
Overall tor Bandwidth: 48828.06 Mbps
Transmission: Success

I've refactored the code, added environmental variables to configure the options, and split it into commits:

Branch: feature14175-chutney-performance-v2 Repository: https://github.com/teor2345/chutney.git

I've also added code to set the environmental variables from command-line arguments to tor/src/test/test-network.sh

Branch: feature14175-chutney-performance-v2 Repository: https://github.com/teor2345/tor.git

The available options are (env variable for chutney, argument for test-network.sh):

CHUTNEY_DATA_BYTES=n sends n bytes per test connection (10 KBytes) --bytes n

CHUTNEY_CONNECTIONS=n makes n test connections per client (1) --connections n

CHUTNEY_HS_MULTI_CLIENT=1 makes each client connect to each HS (0) --hs-multi-client 1

When enough data is transmitted, chutney verify reports:

Single Stream Bandwidth: the speed of the slowest stream, end-to-end

Overall tor Bandwidth: the sum of the bandwidth across each tor instance This approximates the CPU-bound tor performance on the current machine, assuming everything is multithreaded and network performance is infinite.

Note: using --connections 7 or greater on a HS will trigger a verification failure due to legacy/trac#15937 (moved).

These branches cover this task and all child tasks.

Trac:
Status: assigned to needs_review

I have tested these changes myself on Linux and OS X with the following commands:

src/test/test-network.sh --sleep 30 --bytes 102400000 --flavour bridges+ipv6+hs

Verifying data transmission:
Connecting:
  Exit to 127.0.0.1:4747 via client localhost:9010
  Exit to 127.0.0.1:4747 via client localhost:9011
  HS to jarz6vgn5xbbvnrk.onion:5858 (127.0.0.1:4747) via client localhost:9010
Transmitting Data:
.........................................................
Single Stream Bandwidth: 79.08 MBytes/s
Overall tor Bandwidth: 1265.23 MBytes/s
Transmission: Success

src/test/test-network.sh --sleep 30 --flavour bridges+ipv6+hs --connections 5

Verifying data transmission:
Connecting:
  Exit to 127.0.0.1:4747 via client localhost:9010
  Exit to 127.0.0.1:4747 via client localhost:9011
  HS to t4c3o37sgnc43flm.onion:5858 (127.0.0.1:4747) via client localhost:9010
Transmitting Data:
...............
Transmission: Success

src/test/test-network.sh --sleep 30 --flavour hs-025 --hs-multi-client 1

Verifying data transmission:
Connecting:
  HS to q6hmwsdh5z4kmtoj.onion:5858 (127.0.0.1:4747) via client localhost:9014
  HS to q6hmwsdh5z4kmtoj.onion:5858 (127.0.0.1:4747) via client localhost:9015
  HS to q6hmwsdh5z4kmtoj.onion:5858 (127.0.0.1:4747) via client localhost:9016
  HS to q6hmwsdh5z4kmtoj.onion:5858 (127.0.0.1:4747) via client localhost:9017
  HS to q6hmwsdh5z4kmtoj.onion:5858 (127.0.0.1:4747) via client localhost:9018
  HS to q6hmwsdh5z4kmtoj.onion:5858 (127.0.0.1:4747) via client localhost:9019
  HS to ubz5w5ncwbvvqk75.onion:5858 (127.0.0.1:4747) via client localhost:9014
  HS to ubz5w5ncwbvvqk75.onion:5858 (127.0.0.1:4747) via client localhost:9015
  HS to ubz5w5ncwbvvqk75.onion:5858 (127.0.0.1:4747) via client localhost:9016
  HS to ubz5w5ncwbvvqk75.onion:5858 (127.0.0.1:4747) via client localhost:9017
  HS to ubz5w5ncwbvvqk75.onion:5858 (127.0.0.1:4747) via client localhost:9018
  HS to ubz5w5ncwbvvqk75.onion:5858 (127.0.0.1:4747) via client localhost:9019
  HS to kdgd6wb4w4gnvbrq.onion:5858 (127.0.0.1:4747) via client localhost:9014
  HS to kdgd6wb4w4gnvbrq.onion:5858 (127.0.0.1:4747) via client localhost:9015
  HS to kdgd6wb4w4gnvbrq.onion:5858 (127.0.0.1:4747) via client localhost:9016
  HS to kdgd6wb4w4gnvbrq.onion:5858 (127.0.0.1:4747) via client localhost:9017
  HS to kdgd6wb4w4gnvbrq.onion:5858 (127.0.0.1:4747) via client localhost:9018
  HS to kdgd6wb4w4gnvbrq.onion:5858 (127.0.0.1:4747) via client localhost:9019
  HS to gooa5v7yjkf6fbup.onion:5858 (127.0.0.1:4747) via client localhost:9014
  HS to gooa5v7yjkf6fbup.onion:5858 (127.0.0.1:4747) via client localhost:9015
  HS to gooa5v7yjkf6fbup.onion:5858 (127.0.0.1:4747) via client localhost:9016
  HS to gooa5v7yjkf6fbup.onion:5858 (127.0.0.1:4747) via client localhost:9017
  HS to gooa5v7yjkf6fbup.onion:5858 (127.0.0.1:4747) via client localhost:9018
  HS to gooa5v7yjkf6fbup.onion:5858 (127.0.0.1:4747) via client localhost:9019
  HS to ckn2xwnlouxasn52.onion:5858 (127.0.0.1:4747) via client localhost:9014
  HS to ckn2xwnlouxasn52.onion:5858 (127.0.0.1:4747) via client localhost:9015
  HS to ckn2xwnlouxasn52.onion:5858 (127.0.0.1:4747) via client localhost:9016
  HS to ckn2xwnlouxasn52.onion:5858 (127.0.0.1:4747) via client localhost:9017
  HS to ckn2xwnlouxasn52.onion:5858 (127.0.0.1:4747) via client localhost:9018
  HS to ckn2xwnlouxasn52.onion:5858 (127.0.0.1:4747) via client localhost:9019
Transmitting Data:
..............................
Transmission: Success

Sounds plausible, looks reasonable. Merging!

Please reparent and/or close subtickets as appropriate, then close this?

(Code runs on OS X and Linux, on at least both of my boxes, and Corey's from tor-dev.)

Child tickets all closed.

Trac:
Resolution: N/A to fixed
Status: needs_review to closed

Hi nickm,

Looks like the tor branch for this didn't get merged.

Would you mind merging it?

The details are:

I've also added code to set the environmental variables from command-line arguments to tor/src/test/test-network.sh

Branch: feature14175-chutney-performance-v2 Repository: ​https://github.com/teor2345/tor.git

The available options are (env variable for chutney, argument for test-network.sh):

CHUTNEY_DATA_BYTES=n sends n bytes per test connection (10 KBytes)
--bytes n

CHUTNEY_CONNECTIONS=n makes n test connections per client (1)
--connections n

CHUTNEY_HS_MULTI_CLIENT=1 makes each client connect to each HS (0)
--hs-multi-client 1

Trac:
Resolution: fixed to N/A
Status: closed to reopened

Merged, sorry for missing this!

Trac:
Resolution: N/A to implemented
Status: reopened to closed
Keywords: tor-perf SponsorR SponsorS tor-hs tor-client tor-bridge deleted, tor-perf SponsorR SponsorS tor-hs tor-client tor-bridge TorCoreTeam201507 added

closed

mentioned in issue legacy/trac#15936 (moved)

mentioned in issue legacy/trac#15937 (moved)

mentioned in issue legacy/trac#16013 (moved)

mentioned in issue legacy/trac#16386 (moved)

moved from legacy/trac#14175 (moved)

added Feature label and removed 1 deleted label

added Bridge Client Onion Services labels and removed 1 deleted label