Bump LLVM to 13.0.0 for android builds

added Next label

added Doing label and removed Next label

mentioned in merge request !414 (merged)

some of @pier's breadcrumbs from !405 (closed):

After I switched back to Clang 12, I noticed that when we target WASM, we just copy the clang_rt library to the correct path:
cp /var/tmp/dist/wasi-sysroot/lib/clang/11.0.0/lib/wasi/libclang_rt.builtins-wasm32.a $rtdir
(see tor-browser-build/projects/firefox/build). I do not know whether this would work also for Android, though.

./mach configure stops immediately when it tries to create a simple program to check that the compiler works. The error was:
/var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/arm-linux-androideabi-ld: error: cannot open /var/tmp/dist/clang/lib/clang/13.0.0/lib/linux/libclang_rt.builtins-arm-android.a

here is full error that i get (when trying to just build for x86:

 0:03.77 checking for executable suffix... configure: error: installation or configuration problem: compiler cannot create executables.
 0:03.77 DEBUG: <truncated - see config.log for full output>
 0:03.77 DEBUG: configure:878: /var/tmp/dist/clang/bin/clang -std=gnu99 --target=i686-linux-android -c -fno-short-enums -fno-exceptions  -isystem /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/sysroot/usr/include/i686-linux-android -isystem /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/sysroot/usr/include -gcc-toolchain /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/x86-4.9/prebuilt/linux-x86_64 -D__ANDROID_API__=16  conftest.c 1>&5
 0:03.77 DEBUG: configure:874:8: error: use of undeclared identifier '__MINGW32__'
 0:03.77 DEBUG: return __MINGW32__;
 0:03.77 DEBUG:        ^
 0:03.77 DEBUG: 1 error generated.
 0:03.77 DEBUG: configure: failed program was:
 0:03.77 DEBUG: #line 871 "configure"
 0:03.77 DEBUG: #include "confdefs.h"
 0:03.77 DEBUG:
 0:03.77 DEBUG: int main() {
 0:03.77 DEBUG: return __MINGW32__;
 0:03.77 DEBUG: ; return 0; }
 0:03.77 DEBUG: configure:897: checking for executable suffix
 0:03.77 DEBUG: configure:907: /var/tmp/dist/clang/bin/clang -std=gnu99 --target=i686-linux-android -o conftest -fno-short-enums -fno-exceptions  -isystem /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/sysroot/usr/include/i686-linux-android -isystem /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/sysroot/usr/include -gcc-toolchain /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/x86-4.9/prebuilt/linux-x86_64 -D__ANDROID_API__=16  -L/var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/platforms/android-16/arch-x86/usr/lib -Wl,-rpath-link=/var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/platforms/android-16/arch-x86/usr/lib --sysroot=/var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/platforms/android-16/arch-x86 -gcc-toolchain /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/x86-4.9/prebuilt/linux-x86_64  -fuse-ld=bfd conftest.c  1>&5
 0:03.77 DEBUG: /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android-ld.bfd: cannot find /var/tmp/dist/clang/lib/clang/13.0.0/lib/linux/libclang_rt.builtins-i686-android.a: No such file or directory
 0:03.77 DEBUG: /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android-ld.bfd: cannot find -l:libunwind.a
 0:03.77 DEBUG: /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android-ld.bfd: cannot find /var/tmp/dist/clang/lib/clang/13.0.0/lib/linux/libclang_rt.builtins-i686-android.a: No such file or directory
 0:03.77 DEBUG: /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android-ld.bfd: cannot find -l:libunwind.a
 0:03.77 DEBUG: clang-13: error: linker command failed with exit code 1 (use -v to see invocation)
 0:03.77 DEBUG: configure: error: installation or configuration problem: compiler cannot create executables.
 0:03.77 ERROR: old-configure failed
Creating local state directory: /var/tmp/tmp.VNI9x7XdyS/.mozbuild/srcdirs/geckoview-c025288dbe5b-95d33a1f6213
Error running mach:

    ['configure', '--with-tor-browser-version=testbuild', '--with-java-bin-path=/usr/lib/jvm/java-1.8.0-openjdk-amd64/bin', '--without-wasm-sandboxed-libraries']

The error occurred in code that was called by the mach command. This is either
a bug in the called code itself or in the way that mach is calling it.
You can invoke |./mach busted| to check if this issue is already on file. If it
isn't, please use |./mach busted file configure| to report it. If |./mach busted| is
misbehaving, you can also inspect the dependencies of bug 1543241.

If filing a bug, please include the full output of mach, including this error
message.

The details of the failure are as follows:

Exception: Process executed with non-0 exit code 1: ['/var/tmp/build/geckoview-c025288dbe5b/obj-i386-linux-android/_virtualenvs/build/bin/python', '/var/tmp/build/geckoview-c025288dbe5b/configure.py', '--with-tor-browser-version=testbuild', '--with-java-bin-path=/usr/lib/jvm/java-1.8.0-openjdk-amd64/bin', '--without-wasm-sandboxed-libraries']

  File "/var/tmp/build/geckoview-c025288dbe5b/python/mozbuild/mozbuild/build_commands.py", line 254, in configure
    return driver.configure(
  File "/var/tmp/build/geckoview-c025288dbe5b/python/mozbuild/mozbuild/controller/building.py", line 1528, in configure
    status = self._run_command_in_objdir(
  File "/var/tmp/build/geckoview-c025288dbe5b/python/mozbuild/mozbuild/base.py", line 855, in _run_command_in_objdir
    return self.run_process(cwd=self.topobjdir, **args)
  File "/var/tmp/build/geckoview-c025288dbe5b/python/mach/mach/mixin/process.py", line 174, in run_process
    raise Exception(

the relevant line from which appears to be:

 0:03.77 DEBUG: /var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/bin/i686-linux-android-ld.bfd: cannot find /var/tmp/dist/clang/lib/clang/13.0.0/lib/linux/libclang_rt.builtins-i686-android.a: No such file or directory

or just:

cannot find /var/tmp/dist/clang/lib/clang/13.0.0/lib/linux/libclang_rt.builtins-i686-android.a: No such file or directory

consistent with what @pierov observed.

the only libclang_rt.builtin i can find after the error is for x86_64. from a debug shell:

rbm@rbm:/var/tmp/tmp.VNI9x7XdyS$ ls /var/tmp/dist/clang/lib/clang/13.0.0/lib/linux | grep builtins
libclang_rt.builtins-x86_64.a

i also see that we have the static library file in question (libclang_rt.builtins-i686-android.a) located in another directory in /var/dist :

rbm@rbm:/var/tmp/dist$ find . -name libclang_rt.builtins-i686-android.a
./android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/lib64/clang/9.0.8/lib/linux/libclang_rt.builtins-i686-android.a

this suggests we might be able to simply copy it to the corect location (as per @pierov 's conjectural workaround).

however, it seems concerning that the .a file in question lives in a clang/9.0.8 directory? and we are trying to target 13.0.0?

Correct, we want libclang_rt.builtins-i686-android.a we built from 13.0.0. This reminds me a bit of #40307 (closed) (5c4ec887): https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/master/projects/macosx-toolchain/build#L78

Correct, we want libclang_rt.builtins-i686-android.a we built from 13.0.0.

is it expected that although we apparently already built libclang_rt.builtins-i686-android.a from llvm 13.0.0 it is nowhere to to be found in /var/dist at this point in the build? (ie: no artifacts from prior stages there?)

This reminds me a bit of #40307 (closed) (5c4ec887): http://eweiibe6tdjsdprb4px6rqrzzcsi22m4koia44kc5pcjr7nec2rlxyad.onion/tpo/applications/tor-browser-build/-/blob/master/projects/macosx-toolchain/build#L78

how so?

I tried to (cross-)compile Clang with compiler-rt outside tor-browser-build, but I did not get that file.

However, LLVM also has this page: https://llvm.org/docs/HowToCrossCompileBuiltinsOnArm.html .

They mention a arm-linux-gnueabihf sysroot to use with QEMU in user space, which may be the most difficult part for us... (Though it seems that it is needed only for tests).

Other conjectures:

we can use the crosscompiler of the NDK (requirement: «clang executable with support for the ARM target»)
we could check if there is a way to create (part of) the NDK by ourselves, since the NDK contains the library we need

hmm... i do not think i have enough llvm/clang fu to close this ticket myself.

some options:

someone pairs with me to help me get enough context to push the ticket forward
someone writes a brain dump of their current understanding of the problem to try to accomplish the same
we re-asign the ticket

@sysrqb @pierov -- thoughts?

I have some LLVM/clang experience (I have messed up with libtooling and LLVM compilation in general), and I like this kind of stuff

I am 100% available to follow the guide I have linked, and I have some ideas. However I do not know if I can produce something that is working for our needs

I think that we could at least cross-compile Clang for an ARM host. I mean: install gcc-arm-linux-gnueabihf on a Debian container and use it to compile LLVM/Clang/compiler-rt/libcxx. This must produce a working libclangrt-etc (at least gnueabi! I do not know whether we can compile for Android! But with the NDK we should be able to), but I fear it will increase compilation time by a lot. You could pass only compiler-rt as the project to build, but this does not work...

@aguestuser I wanted to start a build because I thought it would take a lot... But actually it was super fast and I got a library for ARMv8! Probably we can get also the other libraries:

git clone https://github.com/llvm/llvm-project.git
cd llvm-project
mkdir build
cd build
cmake \
  ../compiler-rt/ \
  -DCMAKE_SYSTEM_NAME=Android \
  -DCMAKE_ANDROID_ARCH_ABI=arm64-v8a \
  -DCMAKE_ANDROID_NDK="$PWD/../../android-ndk-r21d/" \
  -DCMAKE_ANDROID_STL_TYPE=c++_static

Credits: https://cmake.org/cmake/help/latest/manual/cmake-toolchains.7.html#cross-compiling-for-android-with-the-ndk

It works with arm64-v8a, x86 and x86_64, but it does not work with armeabi-v7a...

It seems I forgot to mention: look at what Mozilla did is usually a helpful step in this process. For reference: https://hg.mozilla.org/releases/mozilla-beta/rev/dcb91b95f32bb7692169357bc6c8d1cfa81e34c1 (tor-browser@0d12e234)

We might want to follow tor-browser@1263f84c and tor-browser@ef3f58e3, too (after we solve the build issue).

In #40433 (comment 2781925), my comment related to aligning our llvm build config with Mozilla's for macOS.

My current guess is that we must do the same for Android (which I believe is what @pierov is implying, too): build-clang.py and build-compiler-rt.sh compared with our clang build (and see the macOS-specific build here)

In summary, we may not be building compiler-rt for android.

I don't have clarity on the mingw error.

but it does not work with armeabi-v7a...

Lol, it turns out the solution was as simply as setting -DCMAKE_C_FLAGS='-DARMEABI_V7A'.

So, this CMake command line is enough to create our craved libclang_rt.builtins-arm-android.a :

cmake ../compiler-rt/ \
  -DCMAKE_SYSTEM_NAME=Android \
  -DCMAKE_ANDROID_ARCH_ABI=armeabi-v7a \
  -DCMAKE_ANDROID_NDK='.../android-ndk-r21d' \
  -DCMAKE_C_FLAGS='-DARMEABI_V7A' \
  -DCOMPILER_RT_BUILD_BUILTINS=ON \
  -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
  -DCOMPILER_RT_BUILD_MEMPROF=OFF \
  -DCOMPILER_RT_BUILD_PROFILE=OFF \
  -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
  -DCOMPILER_RT_BUILD_XRAY=OFF

Now, from what I can understand from build-clang.py, Mozilla is building rt as a second step (first, they build only for x86_64, then they use it to compile again for all archs).

I have tried to build compiler-rt on my host, rather than a container, and encountered several difficulties.

I am investigating more on that however, to understand if we can do more like Mozilla.

I don't have clarity on the mingw error.

My idea is that it is an expected error (on non-mingw environments). I have the same error in my local builds. A failure simply means that you are not using mingw .

added Next label and removed Doing label

assigned to @pierov and unassigned @aguestuser

removed Next label

added Doing label

Mozilla is planning to switch to Clang 14 with Rust 1.60.0 by the beginning of April: https://bugzilla.mozilla.org/show_bug.cgi?id=1758780

Success! I managed to build geckoview (armv7) with Clang 13!

I also had to add a pair of directives to all mozconfigs.

@boklm: I need to add 4 blocks that are almost all like this to projects/clang/build:

cd ..
mkdir build-compilerrt-$ARCH
cd build-compilerrt-$ARCH
cmake ../compiler-rt/ -G "Unix Makefiles" \
  -DCMAKE_INSTALL_PREFIX=$rtdistdir \
  -DCMAKE_SYSTEM_NAME=Android \
  -DCMAKE_ANDROID_ARCH_ABI=$ARCH \
  -DCMAKE_ANDROID_NDK='/var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d' \
  -DCMAKE_C_FLAGS='-fuse-ld=lld --rtlib=compiler-rt' \
  -DCMAKE_CXX_FLAGS='-fuse-ld=lld --rtlib=compiler-rt' \
  -DCMAKE_EXE_LINKER_FLAGS='-L/var/tmp/dist/android-toolchain/android-sdk-linux/ndk/android-ndk-r21d/toolchains/llvm/prebuilt/linux-x86_64/$ARCH_BUT_DIFFERENT-linux-android/lib' \
  -DCOMPILER_RT_BUILD_BUILTINS=ON \
  -DCOMPILER_RT_BUILD_LIBFUZZER=OFF \
  -DCOMPILER_RT_BUILD_MEMPROF=OFF \
  -DCOMPILER_RT_BUILD_PROFILE=OFF \
  -DCOMPILER_RT_BUILD_SANITIZERS=OFF \
  -DCOMPILER_RT_BUILD_XRAY=OFF
make -j[% c("buildconf/num_procs") %]
make install

Also, armv7 needs -DARMEABI_V7A in both flags for C and C++ compilers.

Do you have any suggestion to avoid copying and pasting the same block four times in the file?

Probably, since it is a bash script, I could just create a bash function...

(assuming we want to continue with this workaround to build with the NDK)

Also, armv7 needs -DARMEABI_V7A in both flags for C and C++ compilers.

Do you have any suggestion to avoid copying and pasting the same block four times in the file?

Probably, since it is a bash script, I could just create a bash function...

What is the part that is different between the 4 blocks?

If that's just the directory, maybe a for loop can work:

for component in compiler-rt component2 component3
do
  mkdir build-$component-$ARCH
  cd build-$component-$ARCH
  cmake ../$component/ -G "Unix Makefiles" \
  ...
done

If that's just the directory, maybe a for loop can work

No, I would need 3 possible parameters: architectures with two variants (e.g., I need to have both x86 and i686), and additional flags for the compiler (used only by 32-bit ARM).

I have found that it is possible to iterate over tuples, but having the function seems more clean to me at this point .

marked this issue as related to #40446 (closed)

added Q2 label

The rebase for v99 includes Clang 13.0.1, however:

it builds compiler-rt with the NDK's compiler, instead of using the just compiled one
its linker is not working on x86/x86_64
- it cannot link some libraries that contain compressed symbols (I found a similar issue on the NDK)
- we have never used lld to link Geckoview anyway (I think we should, but I cannot understand what Mozilla is currently doing, and in any case it may be a problem of $PATH priority and/or build environments)

We have merged the support for Clang 13.0.1 in !438 (merged), but these problems remain.

My proposal is to open a new issue and address them when switching to Clang 14 (that will be needed for v101).

We have merged the support for Clang 13.0.1 in !438 (merged), but these problems remain.

My proposal is to open a new issue and address them when switching to Clang 14 (that will be needed for v101).

This sounds like a good plan.

Cool, the new issue is #40480 (closed).

mentioned in merge request !438 (merged)

mentioned in issue #40480 (closed)

closed

mentioned in commit shelikhoo/tor-browser-build@35ac70b0

mentioned in issue #40582 (closed)

Bump LLVM to 13.0.0 for android builds

context

changes

Child items 0

Activity