Creating incremental MAR files for the hardened builds is broken

ASan seems to break the creation of incremental MAR files. mbsdiff is basically frozen while trying to cope with libxul. Using a non-ASan MAR tools archive is working.

added GeorgKoppen201512 in Legacy / Trac TorBrowserTeam201608R in Legacy / Trac component::applications/tor browser in Legacy / Trac owner::mcs in Legacy / Trac priority::medium in Legacy / Trac resolution::fixed in Legacy / Trac severity::normal in Legacy / Trac status::closed in Legacy / Trac tbb-gitian in Legacy / Trac tbb-hardened in Legacy / Trac type::defect in Legacy / Trac labels

Does mbsdiff appear to be in a CPU intensive loop? Or maybe is it using an excessive amount of virtual memory?

It sounds like the non-ASan mbsdiff is working and can produce a diff from two ASan-built copies of libxul? That implies that the problem is not just something like "mbsdiff cannot handle really large binaries" but some flaw in mbsdiff that is caused by it being compiled with ASan.

Trac:
Cc: boklm to boklm, brade, mcs

Does mbsdiff appear to be in a CPU intensive loop? Or maybe is it using an excessive amount of virtual memory?

Yes, it appears to be in a CPU intensive loop, with no IO-wait. It is also using an excessive amount of virtual memory (top says 20.001t).

Replying to boklm:

Does mbsdiff appear to be in a CPU intensive loop? Or maybe is it using an excessive amount of virtual memory?

Yes, it appears to be in a CPU intensive loop, with no IO-wait. It is also using an excessive amount of virtual memory (top says 20.001t).

Increased memory and CPU use a known issue with ASan builds.

Trac:
Keywords: N/A deleted, GeorgKoppen201512 added

gk, please let me know if you need Kathy and me to look at this. I suppose the first step is to run the mbsdiff command under gdb and see what it is doing. It is possible that diffing libxul just causes a bunch of CPU and memory to be consumed, which is made much worse by ASan.

Also see http://www.daemonology.net/bsdiff/ for some info about expected memory usage.

Tickets for Jan 2016.

Trac:
Keywords: TorBrowserTeam201512 deleted, TorBrowserTeam201601 added

Trac:
Status: new to assigned
Owner: tbb-team to mcs

Putting stuff on the radar for February.

Trac:
Keywords: TorBrowserTeam201601 deleted, TorBrowserTeam201602 added

Trac:
Keywords: TorBrowserTeam201602 deleted, TorBrowserTeam201603 added

Trac:
Keywords: TorBrowserTeam201603 deleted, TorBrowserTeam201604 added

Moving tickets

Trac:
Keywords: TorBrowserTeam201604 deleted, TorBrowserTeam201605 added

Trac:
Keywords: TorBrowserTeam201605 deleted, TorBrowserTeam201606 added

Trac:
Keywords: TorBrowserTeam201606 deleted, TorBrowserTeam201607 added

Moving items to August 2016.

Trac:
Keywords: TorBrowserTeam201607 deleted, TorBrowserTeam201608 added

My ASan-enabled mbsdiff ran for hours without completing while trying to generate diffs for libXUL (eventually I gave up and killed it). The problem appeared to be CPU usage, not VM exhaustion.

I think the simplest solution is to avoid using ASan when generating the MAR files. Here is a patch: https://gitweb.torproject.org/user/brade/tor-browser.git/commit/?h=bug17858-01&id=2126d94c8191880bdb20370566c5acc13b817ff3

When I tested this, my system took 187 seconds to finish a 'make incrementals-alpha' command and produced a tor-browser-linux64-6.5a1-hardened-6.5a2-hardened_ALL.incremental.mar file of size 23MB (the full MAR file is 110MB).

Trac:
Keywords: TorBrowserTeam201608 deleted, TorBrowserTeam201608R added
Reviewer: N/A to N/A
Status: assigned to needs_review

Nice find! This is fixed with commit 37e6040d on tor-browser-45.3.0esr-6.5-1.

Trac:
Resolution: N/A to fixed
Status: needs_review to closed

closed

moved from legacy/trac#17858 (moved)

added Bug label and removed 1 deleted label

added 1 deleted label and removed 1 deleted label

added Security label and removed 1 deleted label