Keep Good Summaries From Reliable Sources. Example: Tor's CODEC Strategy, zlib, lzma, zstd
Ironically, about a week after I spent a bunch of time reading up on zlib, lzma, and zstd, on 2019-09-13 someone (Steve Snyder) on the tor-dev list asked: Given the multiple compression types supported (none, lzma, zlib, zstd), what is the order of preference for runtime use?
Put another way, which compression method(s) should be supported to get optimal runtime performance from a Tor node?
To which Nick Mathewson replied: For big objects like consensuses or consensus diffs that are sent over and over, relays prefer to use whichever compression method has the highest compression -- that's lzma2, then zstd, then zlib, then none. Lzma2 (aka xz) is more expensive to calculate, but the relays only need to calculate it once per compressed object, and then they can send it over and over.
For smaller objects that are compressed in a stream (descriptors and microdescriptors), relays will not use xz, since it would be to expensive to recompute it for every stream. They'll prefer zstd, then zlib, then none.
So if you want to save bandwidth above all, you should enable all compression algorithms.
If you want to save CPU above all, you should enable all compression algorithms except xz.
If you want to save bandwidth and CPU, I think enabling all the compression algorithms will result in Tor making good choices (as described above). But I'd appreciate benchmarks if anybody has tried it both ways to find out.
This awesome summary, which I'd not gleaned in all my reading, should be preserved some place in a hierarchical structure of a builder/developer FAQ.
As you know, by default Tor reports via standard output and logging, the compression types compiled into Tor. I wanted to know if it was worth the effort to add more than what the current default is.
There didn't seem a place to put it in the existing structure. Both https://2019.www.torproject.org/docs/tor-doc-unix.html.en and https://2019.www.torproject.org/docs/tor-doc-osx.html.en are places where it makes sense, but best I can tell this raises a few issues: duplicating info is not great, those pages are official, and from other pages on TorProject.org I've read, those pages are output from some other software. And looking at the trac index page https://trac.torproject.org/projects/tor/wiki/TitleIndex is overwhelming, I'm not sure this awesome tidbit should be yet another page on that list.
Please excuse me if I placed this under the wrong Type, and also I wasn't sure what Subcomponent should be chosen.