Firefox and Tor Browser hardening remains a very open area. Let's use this page to keep track of our questions and efforts.
For motivation, see the table of historical Firefox vulnerabilities.
Memory hardening options
Here's a list of possible hardeners we have or are looking into, and comments.
DieHarder A malloc implementation focused on security. Scrambles the heap in a way that attempts to maximize entropy. Originally demonstrated using Firefox. Here's a patch that builds Tor Browser with DieHarder. Unfortunately the browser crashes pretty frequently, so we're investigating why. Claims good performance.
Address Sanitizer ASan was deployed in tbb-hardened for building both Tor and tor-browser. However, ASan degrades performance and is claimed not suitable for production hardening. It's designed for debugging, and is often easy to circumvent if done intentionally (with the exception of linear buffer overflows). While other hardening features will trickle down to alpha, and eventually stable, ASan will not do so for this reason, unless plans change.
UndefinedBehaviorSanitizer (UBSan) UBSan catches undefined behaviors. -fsanitize=undefined for clang and gcc. Unlike ASan, UBSan is designed for use in production environments as a hardening tool, when used with -fsanitize-undefined-trap-on-error (GCC syntax), a keyword which triggers ud2, telling the CPU to intentionally raise an illegal instruction exception and aborting execution of the program. Using -fsanitize=undefined enables most checks and is very aggressie. The gcc(1) manpage gives a list of valid keywords for enabling individual suboptions.
SoftBounds+CETS This memory hardening tool provides full memory safety to C programs compiled with clang. (Although requires some TLC for hardening nontrivial programs.) In principle it should also work for C++ programs, but this has not yet been fully implemented. Performance overhead claims vary, but might be as much as 100%.
CPI/CPS CPI (code pointer integrity) is claimed to provide memory safety for code pointers. Code Pointer Separation (CPS) is a slightly less strict hardener that is also claimed to provide good protection. Require clang. Both claim to have low performance overheads by concentrating on code pointers but not SoftBounds+CETS. Source code has not yet been released (as far as I can tell), and protection against UAF was not implemented as of the time of publication (2014).
-fstack-protector, -fstack-protector-all, -fstack-protector-strong gcc compiler flags. The last is apparently used in ChromeOS, IIUC. Places canaries in the stack to detect buffer overflows. -fstack-protector protects functions that call alloca(), and functions with buffers larger than 8 bytes (this size can be overridden by -fssp-buffer-size=$nbytes). -fstack-protector-strong additionally protects functions that have local array definitions, and functions that have references to local frame addresses. -fstack-protector-all, like the name suggests, protects all functions.
PartitionAlloc A malloc replacement that allows you to partition allocations by size or location. Currently used by Chrome. Investigated for use with Firefox by Guilherme P. Gonçalves (Part 1, Part 2) and tjr. See also HardenedPartitionAlloc.
jemalloc Firefox currently bundles jemalloc and it is possible to use it as the memory allocator with the right flags. In #10281 we have enabled redzones as a crude hardening measure. tjr has also experimented with allocating objects in random partition by using jemalloc's arenas. Note that redzones will be removed in jemalloc 5. See also shadow, a "jemalloc heap exploitation framework", documented here and here.
RAP RAP (Reuse Attack Protector) protects against ROP and all other forms of code-reuse attacks (SROP, BROP, LOP, etc). However, it requires that all uses of function pointers correctly match their prototypes. Any mismatches will result in the program being killed. Unfortunately many programs do abuse function pointers and need to be patched in order to become compatible with RAP. In a discussion with pipacs, the developer of RAP, it became clear that Firefox had two large structures internally that were so badly damaged and convoluted that it would be utterly impossible to untangle the mess and make it compatible with RAP without rewriting massive amounts of core code. Even without that nearly insurmountable problem, it would require a patch many megabytes in size to correct all the function pointer abuse which a program as large and as poorly written as Firefox contains. As nice as RAP is, for now, we should conclude that defending from ROP completely is just not possible with this codebase.
Ironclad C++ "A library-augmented type-safe subset of C". This tool carries out a semi-automated conversion of all bare pointers to smart pointers. Probably not feasible for a huge codebase like Firefox, but maybe vulnerable things, like tor or modules within Firefox?
selfrando Randomizes the code layout. Selfrando has been integrated with tbb-hardened. Source code here. Selfrando is subject to infoleaks and has many problems, but it isn't known to cause problems, so it's better than nothing.
Code Flow Integrity Implemented in clang as -fsanitize=cfi. See #20361