unit-test fail to cross-compile for aarch64

When trying to cross-compile tor for AARCH64 following error is given: src/common/libor-crypto-testing.a(src_common_libor_crypto_testing_a-crypto_format.o): In function crypto_write_tagged_contents_to_file': crypto_format.c:(.text+0x1c): relocation truncated to fit: R_AARCH64_LDST64_ABS_LO12_NC against symbol smartlist_new' defined in .text section in src/common/libor.a(container.o) collect2: error: ld returned 1 exit status

For complete log see here: http://autobuild.buildroot.net/results/13ef47c962afefbaa9ea7a95de083f885f1a8825/

The error might be misleading, I was told that this can happen because test and non-test library archives are getting mixed up and the aarch64 toolchain is more picky about this then other toolchains.

Trac:
Username: wbx

changed milestone to %Tor: 0.2.7.x-final

added 027-backport aarch64 component::core tor/tor milestone::Tor: 0.2.7.x-final owner::cypherpunks priority::medium reporter::wbx resolution::fixed severity::normal status::closed tests type::defect unit version::tor 0.2.7.6 labels

It also happens with ARM toolchain when you enable LTO: /home/wbx/tor-musl/toolchain_qemu-arm-vexpress-a9_glibc_arm_hard_eabihf/usr/bin/arm-openadk-linux-gnueabihf-gcc -fwrapv -fno-ident -fhonour-copts -mcpu=cortex-a9 -mfpu=vfpv3-fp16 -mfloat-abi=hard -flto -Os -pipe -fomit-frame-pointer -fno-unwind-tables -fno-asynchronous-unwind-tables -marm -fasynchronous-unwind-tables -Wall -fno-strict-aliasing -L/home/wbx/tor-musl/target_qemu-arm-vexpress-a9_glibc_arm_hard_eabihf/lib -L/home/wbx/tor-musl/target_qemu-arm-vexpress-a9_glibc_arm_hard_eabihf/usr/lib -Wl,-O1 -Wl,-rpath -Wl,/usr/lib -Wl,-rpath-link -Wl,/home/wbx/tor-musl/target_qemu-arm-vexpress-a9_glibc_arm_hard_eabihf/usr/lib -flto -z relro -z now -rdynamic -o src/test/test-memwipe src/test/src_test_test_memwipe-test-memwipe.o src/or/libtor-testing.a src/common/libor-testing.a src/common/libor-crypto-testing.a src/common/libcurve25519_donna.a src/ext/ed25519/ref10/libed25519_ref10.a src/ext/ed25519/donna/libed25519_donna.a src/common/libor.a src/common/libor-event-testing.a src/trunnel/libor-trunnel-testing.a -lz -lm -levent -lssl -lcrypto -lpthread -ldl src/common/container.h:31:1: error: function 'smartlist_free' redeclared as variable MOCK_DECL(void, smartlist_free, (smartlist_t *sl)); ^ src/common/container.c:44:1: note: previously declared here MOCK_IMPL(void, ^ src/common/container.h:30:1: error: function 'smartlist_new' redeclared as variable MOCK_DECL(smartlist_t *, smartlist_new, (void)); ^ src/common/container.c:31:1: note: previously declared here MOCK_IMPL(smartlist_t *, ^ lto1: fatal error: errors during merging of translation units compilation terminated.

The error is a little bit more visible.

Trac:
Username: wbx

Patch is coming, but needs some more testing first.

Trac:
Owner: N/A to cypherpunks
Status: new to assigned

Trac:
0001-Do-not-link-tests-to-both-types-of-libraries.patch

I was able to reproduce the issue by configuring with CFLAGS="-O2 -flto". The attached patch fixed the issue for me. @wbx can you verify this?

As mentioned in the commit message, removing the non-testing library from the test binary dependencies caused undefined reference errors which were caused by missing dependencies. I have added the dependencies that the compiler complained about but i am not 100% sure i got them all because of the optional external dependencies that tor has. For instance, during testing i had to add sandbox.c for one machine because it had sandboxing capabilities while another machine did not (yay for conditional compilation).

Trac:
Status: assigned to needs_review

Trac:
Reviewer: N/A to N/A
Keywords: unit tests aarch64 deleted, unit tests aarch64 027-backport added
Milestone: N/A to Tor: 0.2.8.x-final

Lifting all the other stuff into libor-crypto.a is not really a great idea; those libraries are truly supposed to be separate.

I think that the problem that dgoulet tried to fix in 2dc5d790d314beb202dfa1659161113f1f6dabf1 can be resolved better by making sure that libor-testing.a appears after libor-crypto-testing.a, which depends on it. I'm going to try that fix. If it doesn't work, then probably the right move is to take the dependencies in libor-testing.a and move them into a third library that both libcrypto-testing.a and libor-testing.a.

Possible fix committed as 82df3e70ac191a2a3cabf3574839e7561e00e370. Let's see if the builders like it. Let's close this ticket if it solves everybody's problems, and not otherwise.

Trac:
Status: needs_review to needs_information

Replying to nickm:

Possible fix committed as 82df3e70ac191a2a3cabf3574839e7561e00e370. Let's see if the builders like it. Let's close this ticket if it solves everybody's problems, and not otherwise. This solved the errors that i reproduced and is a lot cleaner than the patch i submitted. Thanks for that.

After looking into the Tor related commits made at Buildroot, i noticed they have now disabled unit tests (see [https://git.busybox.net/buildroot/commit/?id=41a14f3e394f46bd098daf4a76283315e3ae452a]) to (probably?) work around the issues in this ticket. For this reason, i am in favor of backporting the change to 0.2.7 so Buildroot can revert their change and run the unit tests again.

It would also help to get some feedback from wbx.

Marking for possible backport.

Trac:
Status: needs_information to needs_revision
Milestone: Tor: 0.2.8.x-final to Tor: 0.2.7.x-final

Since this is a build break it seems reasonable to consider for backport, and since it only affects the test suite it seems safe. Cherry-picked 82df3e70ac191a2a3cabf3574839e7561e00e370 to maint-0.2.7 and closing this.

Trac:
Resolution: N/A to fixed
Status: needs_revision to closed

closed

moved to tpo/core/tor#18490 (closed)