Skip to content

GitLab

Trac
Trac

GitLab is used only for code review, issue tracking and project management. Canonical locations for source code are still https://gitweb.torproject.org/ https://git.torproject.org/ and git-rw.torproject.org.

Closed (moved)
Opened by Nick Mathewson@nickm🥄

Our zlib deflate objects require lots of memory per directory connection

Compressing descriptors on the fly requires a lot of memory with zlib's default settings. According to zconf.h:

/* The memory requirements for deflate are (in bytes):
            (1 << (windowBits+2)) +  (1 << (memLevel+9))
 that is: 128K for windowBits=15  +  128K for memLevel = 8  (default values)
 plus a few kilobytes for small objects.

We are using windowbits == 15 and memLevel == 8. This might explain in part why busy directories are so memory-hungry. In addition to our investigations for #11787 (moved), we should see whether we can safely reduce this to 32K or so with windowBits == 12 and memlevel == 5. It appears in preliminary investigation that this compression level would not hurt compression very much (0-5% for microdescs, 3-7% for server descriptors).

We could also choose our window size and memlevel based on current memory availability.

To upload designs, you'll need to enable LFS and have admin enable hashed storage. More information