Onionperf issueshttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues2021-04-22T12:33:59Zhttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/40012Avoid traceback when visualizing only unsuccessful measurements2021-04-22T12:33:59ZKarsten LoesingAvoid traceback when visualizing only unsuccessful measurements@asn had a configuration error related to port forwarding, and when he attempted to visualize those measurements, he received weird tracebacks:
```
$ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
2020-11-...@asn had a configuration error related to port forwarding, and when he attempted to visualize those measurements, he received weird tracebacks:
```
$ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
2020-11-03 15:51:31 1604411491.540736 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz
2020-11-03 15:51:31 1604411491.577864 [onionperf] [INFO] done!
2020-11-03 15:51:31 1604411491.586845 [onionperf] [INFO] NumExpr defaulting to 8 threads.
/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding.
/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py:251: UserWarning: Attempting to set identical left == right == -1e-06 results in singular transformations; automatically expanding.
Traceback (most recent call last):
File "/user/.local/bin/onionperf", line 4, in <module>
__import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script
exec(script_code, namespace, namespace)
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module>
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 48, in plot_all
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 205, in __plot_throughput_ecdf
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 235, in __draw_ecdf
File "/usr/lib/python3/dist-packages/pandas/core/frame.py", line 5000, in dropna
raise KeyError(list(np.compress(check, subset)))
KeyError: ['mbps']
$ onionperf visualize --data onionperf.analysis.json.xz "Test Measurements"
2020-11-03 15:50:03 1604411403.946028 [onionperf] [INFO] loading analysis results from /user/tmp/onionperf/analysis/onionperf.analysis.json.xz
2020-11-03 15:50:03 1604411403.976088 [onionperf] [INFO] done!
Traceback (most recent call last):
File "/user/.local/bin/onionperf", line 4, in <module>
__import__('pkg_resources').run_script('OnionPerf==0.8', 'onionperf')
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 650, in run_script
self.require(requires)[0].run_script(script_name, ns)
File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1453, in run_script
exec(script_code, namespace, namespace)
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 622, in <module>
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 382, in main
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/EGG-INFO/scripts/onionperf", line 522, in visualize
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 38, in plot_all
File "/user/.local/lib/python3.8/site-packages/OnionPerf-0.8-py3.8.egg/onionperf/visualization.py", line 129, in __extract_data_frame
KeyError: '44028'
```
We should add another check or two to ensure that there's something to visualize before running into those tracebacks. I'll give this a try today.Ana CusturaAna Custurahttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/40004Include error_code column in CSV output regardless of whether data contains m...2020-12-13T21:59:02ZKarsten LoesingInclude error_code column in CSV output regardless of whether data contains measurements with error codeAs the title says, let's include the `error_code` column in the CSV output regardless of whether data contains measurements with error code or not. This might simplify a few checks in the visualization code to see whether this column exi...As the title says, let's include the `error_code` column in the CSV output regardless of whether data contains measurements with error code or not. This might simplify a few checks in the visualization code to see whether this column exists.https://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/40008Add a --version switch to OnionPerf2020-12-13T21:02:31ZAna CusturaAdd a --version switch to OnionPerfIt would be nice to see the OP version by running something like `onionperf --version`.It would be nice to see the OP version by running something like `onionperf --version`.Karsten LoesingKarsten Loesinghttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/40006Do some repository housekeeping2020-07-24T14:33:44ZKarsten LoesingDo some repository housekeepingWe have moved around quite a few pieces in this repository over the past months. Can we look at the files we didn't touch yet and decide what to do with them? This is going to make help new contributors find their way around the reposito...We have moved around quite a few pieces in this repository over the past months. Can we look at the files we didn't touch yet and decide what to do with them? This is going to make help new contributors find their way around the repository.
### Vagrantfile
Do we still need this file? It seems we're not maintaining it (with `debian/stretch64` as VM box), so maybe we should kill it?
### debian/
It seems like these files were an earlier attempt to get OnionPerf into Debian. But I don't think we're still planning to do that, right? Can we remove this directory and all contained files?
### run_tests.sh
Can we include the single line in `README.md` and adapt it to the way we set up everything else, that is, with `venv`? This file seems a bit lost, covering just running tests out of all other development/release steps.
### .gitignore
There are a few entries in `.gitignore` that we probably don't need anymore:
```
/.onionperf/* # <- remove
onionperf-data # <- either remove, or add onionperf-private
venv # <- keep
*.json.xz # <- keep
*.pdf # <- keep, and add *.csv
*.pyc # <- keep
onionperf/docs/_build # <- remote
.coverage # <- keep
```
### Python file headers
Can we add another line to the Python file headers? (Or more lines as needed?)
```
'''
OnionPerf
Authored by Rob Jansen, 2015
Copyright 2015-2020 The Tor Project # <- new line
See LICENSE for licensing information
'''
```
@robgjansen, @acute, @phw: What do you think about these suggestions?https://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/40003Rewrite README_JSON.md to be a proper specification document2020-07-16T20:19:33ZKarsten LoesingRewrite README_JSON.md to be a proper specification documentWe should rewrite `README_JSON.md` to be a proper specification document, not an annotated example. Maybe it's going to be compact enough to become part of `README.md`, but that's not a hard requirement.We should rewrite `README_JSON.md` to be a proper specification document, not an annotated example. Maybe it's going to be compact enough to become part of `README.md`, but that's not a hard requirement.Ana CusturaAna Custurahttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/40002Write generated torrc files to disk2020-07-02T08:23:27ZKarsten LoesingWrite generated torrc files to diskLet's consider writing generated `torrc` files to disk just like we're writing TGen model files to disk. This can be important as confirmation that configuration changes are effective and for debugging. It's okay if this file isn't being...Let's consider writing generated `torrc` files to disk just like we're writing TGen model files to disk. This can be important as confirmation that configuration changes are effective and for debugging. It's okay if this file isn't being read by the `tor` process, because we're passing its content as command-line arguments to the `tor` process.