Skip to content

Bug 41142: Complete the toolchain for linux-aarch64

NoisyCoil requested to merge NoisyCoil/tor-browser-build:bug_41142 into main

Merge Info

Related Issues

Backporting

Timeline

  • Immediate: patchset needed as soon as possible
  • Next Minor Stable Release: patchset that needs to be verified in nightly before backport
  • Eventually: patchset that needs to be verified in alpha before backport
  • No Backport (preferred): patchset for the next major stable

(Optional) Justification

  • Emergency security update: patchset fixes CVEs, 0-days, etc
  • Censorship event: patchset enables censorship circumvention
  • Critical bug-fix: patchset fixes a bug in core-functionality
  • Consistency: patchset which would make development easier if it were in both the alpha and release branches; developer tools, build system changes, etc
  • Sponsor required: patchset required for sponsor
  • Other: please explain

Issue Tracking

Review

Request Reviewer

  • Request review from an applications developer depending on modified system:
    • NOTE: if the MR modifies multiple areas, please /cc all the relevant reviewers (since gitlab only allows 1 reviewer)
    • accessibility : henry
    • android : clairehurst, dan
    • build system : boklm
    • extensions : ma1
    • firefox internals (XUL/JS/XPCOM) : ma1
    • fonts : pierov
    • frontend (implementation) : henry
    • frontend (review) : donuts, richard
    • localization : henry, pierov
    • macos : clairehurst, dan
    • nightly builds : boklm
    • rebases/release-prep : boklm, dan, ma1, pierov, richard
    • security : ma1
    • signing : boklm, richard
    • updater : pierov
    • misc/other : pierov, richard

Change Description

This MR completes the toolchain for the linux-aarch64 target. What this means in practice is it adds the linux-aarch64 target to rust (plus some other minor changes). Some notes.

  • As you know from the way rust is built for the macos target (which I could see explicitly from the build script, but I believe something similar may hold for windows), in order to build a rust target for cross-compilation you need the sysroot of that target. This means that rust must be built separately for each linux arch, just like you are doing when building it separately for windows and for macos, and that it is not a no_crosscompile project.
  • One issue I came across with rust is using --set target.aarch64-unknown-linux-gnu.linker=clang like you are doing for all other targets fails to build the project. Specifically, while building the rust standard library for the aarch64-linux-gnu target, the build fails complaining that binaries for the wrong architecture are being linked to. From the error message it is very clear that clang, which is selected as the linker for that target, was trying to use the host's x86_64-unknown-linux-gnu-ld internally as the linker instead of aarch64-unknown-linux-gnu-ld. I also found out that this issue is reproducible in more mundane contexts: if I try to cross-compile rust source code for aarch64-unknown-linux-gnu on x86_64 using as config.toml 
    [target.aarch64-unknown-linux-gnu]
linker = "clang"
    I get a similar linking error. There are two straightforward ways to solve this I believe. The first one is to define a wrapper script around clang which calls the correct -target and so on, similar to what you are doing for the macos target. The second one, which I found to be very effective (so much so that I've been using it since day one of cross-compilation to build the full browser) is to simply --set target.aarch64-unknown-linux-gnu.linker=aarch64-linux-gnu-gcc based on the fact that, by default (at least when not failing to select their target like clang), both gcc and clang call aarch64-linux-gnu-ld as their default linker. This does not require any further dependency in the project because, again, building the aarch64-unknown-linux-gnu rust target already requires the full linux-aarch64 sysroot, which is in the gcc-cross project, so the cross-compiler is imported anyway. And it avoids a wrapper script (which to be honest I didn't even test on its own, let alone on the full browser).
  • In this MR you can see the cross-compilation logic fully at work. Projects which build host tools (python, cmake, etc.), as well as projects which build target binaries but don't need the cross-compiler to be built (i.e. clang), as well as projects that don't build neither host nor target (in the sense of aarch64-linux-gnu) binaries but are still built in containers which must not --add-architecture (i.e. wasi-sysroot) are marked as no_crosscompile. On the other hand, for binutils (which as mentioned in the previous MR is the only project which is both compiled and cross-compiled), one needs to specify which target one wants to build. In this MR we only want the host binutils, so we must tell the other projects to only use the linux target when building for linux, or more specifically for linux-cross. This is the reason for the two target redefinitions (there will be one more in firefox).

How Tested

Tested by building the full browsers using this linux-cross branch. With this MR you will be able to build any project for linux-aarch64 apart from firefox, browser, linux-packages and release (left for a later MR), using

./rbm/rbm build $project --target alpha --target torbrowser-linux-aarch64
Edited by boklm

Merge request reports