Skip to content

Proposal idea: migrate from consensus diffs to compression dictionaries

Currently, if a client knows an older consensus, it can ask a dircache for a compressed consensus diff describing the changes from the older consensus to the latest one.

But we can do better: If you use the old consensus as a compression dictionary for compressing the new consensus (this is something that zstd and brotli both support), the compressed results are significantly smaller than our existing compressed consensus diffs.

There's an internet draft for how to negotiate this with HTTP headers.

I'm putting this in Icebox because everybody's busy right now.

To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information