Find why rlbox hurts reproducibility
We had to disable rlbox at the last minute because the initial build for 12.5a2 was not reproducible.
Disabling rlbox again fixed the problem.
- #40353
- #40758
Activity
mentioned in issue #40715 (closed)
So here's what I know from investigating the difference for the linux64 build:
The differing component is
libxul.so:I disassembled the mismatched copies and diff'd the resulting asm. The vast majority of the differences are small differences in byte offsets off the end of
libxul.so's mapped address space. The other difference is a two blocks of re-ordered instructions in:je_malloc_usable_size:450644c450644 < b15bfc: 0f b6 84 0d 40 96 06 movzbl 0x69640(%rbp,%rcx,1),%eax --- > b15c00: 48 01 c1 add %rax,%rcx 450646,450659c450646,450657 < b15c04: 89 d9 mov %ebx,%ecx < b15c06: 80 e1 07 and $0x7,%cl < b15c09: 0f b6 c9 movzbl %cl,%ecx < b15c0c: 0f a3 c8 bt %ecx,%eax < b15c0f: 73 19 jae b15c2a <je_malloc_usable_size@@xul102+0xf3b4a> < b15c11: 42 8b 44 25 10 mov 0x10(%rbp,%r12,1),%eax < b15c16: 81 4c 05 00 80 00 00 orl $0x80,0x0(%rbp,%rax,1) < b15c1d: 00 < b15c1e: 49 8b ae b0 19 00 00 mov 0x19b0(%r14),%rbp < b15c25: 42 8b 5c 25 0c mov 0xc(%rbp,%r12,1),%ebx < b15c2a: 8d 43 c6 lea -0x3a(%rbx),%eax < b15c2d: 83 f8 f5 cmp $0xfffffff5,%eax < b15c30: 77 1e ja b15c50 <je_malloc_usable_size@@xul102+0xf3b70> < b15c32: 89 d8 mov %ebx,%eax --- > b15c08: 89 d9 mov %ebx,%ecx > b15c0a: 80 e1 07 and $0x7,%cl > b15c0d: 0f b6 c9 movzbl %cl,%ecx > b15c10: 0f a3 c8 bt %ecx,%eax > b15c13: 73 19 jae b15c2e <je_malloc_usable_size@@xul102+0xf3b4e> > b15c15: 42 8b 44 25 10 mov 0x10(%rbp,%r12,1),%eax > b15c1a: 81 4c 05 00 80 00 00 orl $0x80,0x0(%rbp,%rax,1) > b15c21: 00 > b15c22: 49 8b ae b0 19 00 00 mov 0x19b0(%r14),%rbp > b15c29: 42 8b 5c 25 0c mov 0xc(%rbp,%r12,1),%ebx > b15c2e: 8d 43 c6 lea -0x3a(%rbx),%eax > b15c31: 83 f8 f5 cmp $0xfffffff5,%eax 450661,450666c450659,450664 < b15c37: 83 c0 99 add $0xffffff99,%eax < b15c3a: 83 f8 fa cmp $0xfffffffa,%eax < b15c3d: 73 11 jae b15c50 <je_malloc_usable_size@@xul102+0xf3b70> < b15c3f: 83 fb 20 cmp $0x20,%ebx < b15c42: 75 2a jne b15c6e <je_malloc_usable_size@@xul102+0xf3b8e> < b15c44: eb 31 jmp b15c77 <je_malloc_usable_size@@xul102+0xf3b97> --- > b15c36: 89 d8 mov %ebx,%eax > b15c38: 83 c8 20 or $0x20,%eax > b15c3b: 83 c0 99 add $0xffffff99,%eax > b15c3e: 83 f8 fa cmp $0xfffffffa,%eax > b15c41: 73 0d jae b15c50 <je_malloc_usable_size@@xul102+0xf3b70> > b15c43: 83 fb 20 cmp $0x20,%ebx 450668c450666 < b15c4d: 00 00 00 --- > b15c48: eb 2d jmp b15c77 <je_malloc_usable_size@@xul102+0xf3b97>1,36c1,36 < b15f08: 83 e2 1f and $0x1f,%edx < b15f0b: 48 09 ca or %rcx,%rdx < b15f0e: 0f b6 8c 13 40 96 06 movzbl 0x69640(%rbx,%rdx,1),%ecx < b15f15: 00 < b15f16: 24 07 and $0x7,%al < b15f18: 0f b6 c0 movzbl %al,%eax < b15f1b: 0f a3 c1 bt %eax,%ecx < b15f1e: 72 41 jb b15f61 <je_malloc_usable_size@@xul102+0xf3e81> < b15f20: 41 f7 c5 00 01 00 00 test $0x100,%r13d < b15f27: 74 18 je b15f41 <je_malloc_usable_size@@xul102+0xf3e61> < b15f29: 42 8b 44 23 0c mov 0xc(%rbx,%r12,1),%eax < b15f2e: 8d 48 c6 lea -0x3a(%rax),%ecx < b15f31: 83 f9 f5 cmp $0xfffffff5,%ecx < b15f34: 77 2b ja b15f61 <je_malloc_usable_size@@xul102+0xf3e81> < b15f36: 83 c8 20 or $0x20,%eax < b15f39: 83 c0 99 add $0xffffff99,%eax < b15f3c: 83 f8 f9 cmp $0xfffffff9,%eax < b15f3f: 77 20 ja b15f61 <je_malloc_usable_size@@xul102+0xf3e81> < b15f41: 41 f7 c5 00 02 00 00 test $0x200,%r13d < b15f48: 0f 84 b2 fd ff ff je b15d00 <je_malloc_usable_size@@xul102+0xf3c20> < b15f4e: 42 8b 44 23 0c mov 0xc(%rbx,%r12,1),%eax < b15f53: 83 f8 09 cmp $0x9,%eax < b15f56: 74 09 je b15f61 <je_malloc_usable_size@@xul102+0xf3e81> < b15f58: 83 f8 20 cmp $0x20,%eax < b15f5b: 0f 85 9f fd ff ff jne b15d00 <je_malloc_usable_size@@xul102+0xf3c20> < b15f61: 42 8b 44 23 14 mov 0x14(%rbx,%r12,1),%eax < b15f66: 45 89 b7 c4 1b 00 00 mov %r14d,0x1bc4(%r15) < b15f6d: 48 83 c4 08 add $0x8,%rsp < b15f71: 5b pop %rbx < b15f72: 41 5c pop %r12 < b15f74: 41 5d pop %r13 < b15f76: 41 5e pop %r14 < b15f78: 41 5f pop %r15 < b15f7a: 5d pop %rbp < b15f7b: c3 retq < b15f7c: 0f 1f 40 00 nopl 0x0(%rax) --- > b15f08: 0f b6 d2 movzbl %dl,%edx > b15f0b: 48 81 ca e0 95 06 00 or $0x695e0,%rdx > b15f12: 48 01 ca add %rcx,%rdx > b15f15: 0f b6 0c 13 movzbl (%rbx,%rdx,1),%ecx > b15f19: 24 07 and $0x7,%al > b15f1b: 0f b6 c0 movzbl %al,%eax > b15f1e: 0f a3 c1 bt %eax,%ecx > b15f21: 72 41 jb b15f64 <je_malloc_usable_size@@xul102+0xf3e84> > b15f23: 41 f7 c5 00 01 00 00 test $0x100,%r13d > b15f2a: 74 18 je b15f44 <je_malloc_usable_size@@xul102+0xf3e64> > b15f2c: 42 8b 44 23 0c mov 0xc(%rbx,%r12,1),%eax > b15f31: 8d 48 c6 lea -0x3a(%rax),%ecx > b15f34: 83 f9 f5 cmp $0xfffffff5,%ecx > b15f37: 77 2b ja b15f64 <je_malloc_usable_size@@xul102+0xf3e84> > b15f39: 83 c8 20 or $0x20,%eax > b15f3c: 83 c0 99 add $0xffffff99,%eax > b15f3f: 83 f8 f9 cmp $0xfffffff9,%eax > b15f42: 77 20 ja b15f64 <je_malloc_usable_size@@xul102+0xf3e84> > b15f44: 41 f7 c5 00 02 00 00 test $0x200,%r13d > b15f4b: 0f 84 af fd ff ff je b15d00 <je_malloc_usable_size@@xul102+0xf3c20> > b15f51: 42 8b 44 23 0c mov 0xc(%rbx,%r12,1),%eax > b15f56: 83 f8 09 cmp $0x9,%eax > b15f59: 74 09 je b15f64 <je_malloc_usable_size@@xul102+0xf3e84> > b15f5b: 83 f8 20 cmp $0x20,%eax > b15f5e: 0f 85 9c fd ff ff jne b15d00 <je_malloc_usable_size@@xul102+0xf3c20> > b15f64: 42 8b 44 23 14 mov 0x14(%rbx,%r12,1),%eax > b15f69: 45 89 b7 c4 1b 00 00 mov %r14d,0x1bc4(%r15) > b15f70: 48 83 c4 08 add $0x8,%rsp > b15f74: 5b pop %rbx > b15f75: 41 5c pop %r12 > b15f77: 41 5d pop %r13 > b15f79: 41 5e pop %r14 > b15f7b: 41 5f pop %r15 > b15f7d: 5d pop %rbp > b15f7e: c3 retq > b15f7f: 90 nopGiven the size of the
je_malloc_usable_sizefunction source and the size of the block in the asm, this is probably just somewhere in the je_malloc implementation rather than that exact function in je_malloc.I've uploaded the relevant differing files here:
https://people.torproject.org/~richard/bugs/tor-browser-build-40750/
added Backlog label
added Task label
marked this issue as related to #40353 (closed)
mentioned in issue #40353 (closed)
added Build System label
Clue 1: the wasi-sysroot project itself isn't built reproducibly at the moment.
wasi-libcuses wildcards in theMakefile. Therefore, while.ofiles are the same, they have a different order inshare/wasi-sysroot/lib/wasm32-wasi/libc.a.So, first step is extracting the .a and rebuilding it, with sorted inputs.
If this is the root cause, we should open an issue upstream (well, we could do it even if it isn't the root cause).
mentioned in merge request !672 (merged)
assigned to @pierov
removed Backlog label
added Doing label
removed Doing label
marked this issue as related to #40758 (closed)
marked this issue as related to #40802 (closed)
mentioned in issue #40802 (closed)
