Strip "__.SYMDEF*" before re-archiving in combine_libs on macOS and iOS.

This patch changes how combine_libs works on Darwin like platforms to make sure we don't include any __.SYMDEF and __.SYMDEF SORTED symbols on the archive before we repack and run ${RANLIB} on the archive.

See: #40683 (closed).

assigned to @ahf

CC @nickm, @mikeperry for testing?

@tla could I get you to test this on x86-64 macOS? I don't have such device anymore :-(

This branch builds fine on x86-64 MacOS; tests still pass.

Sweet, @mikeperry - thank you! @tla no need to test anything unless you discover something breaking here.

I tried it anyway, out of curiosity, and so you can feel even better!

On a MacBook Pro 15-inch, 2018, with an Intel Core i9, 2.9 GHz 6-core, running macOS Monterey 12.6, this works:

mkdir ahf-tor
cd ahf-tor

git clone https://gitlab.torproject.org/ahf/tor.git
git clone https://github.com/libevent/libevent.git
git clone https://github.com/openssl/openssl.git

cd libevent
git checkout release-2.1.12-stable
./autogen.sh
./configure --disable-openssl
make
make install

cd ../openssl
git checkout OpenSSL_1_1_1r
./config
make depend
make build_libs

cd ../tor
git checkout ahf/40683
git submodule update --init --recursive
./autogen.sh
./configure --disable-asciidoc
make

But running make test yields 2 errors out of 1461 tests:

link-handshake/recv_certs/expired_rsa_id: [forking] 
  FAIL src/test/test_link_handshake.c:838: expected log to contain "Certificate already expired"  Captured logs:
    1. info: "TLS error while checking a certificate: bad signature (in rsa routines:int_rsa_verify:---)\n"
    2. info: "TLS error while checking a certificate: EVP lib (in asn1 encoding routines:ASN1_item_verify:---)\n"
    3. info: "Received a bad CERTS cell: The ID certificate was not valid\n"
    4. info: "Received a bad CERTS cell on OR connection (handshaking (Tor, v3 handshake)) with [scrubbed]: Invalid certificate chain!\n"

  [recv_certs/expired_rsa_id FAILED]
link-handshake/recv_certs/expired_rsa_id_ed25519: [forking] 
  FAIL src/test/test_link_handshake.c:838: expected log to contain "Certificate already expired"  Captured logs:
    1. info: "TLS error while checking a certificate: bad signature (in rsa routines:int_rsa_verify:---)\n"
    2. info: "TLS error while checking a certificate: EVP lib (in asn1 encoding routines:ASN1_item_verify:---)\n"
    3. info: "Received a bad CERTS cell: The legacy RSA ID certificate was not valid\n"
    4. info: "Received a bad CERTS cell on OR connection (handshaking (Tor, v3 handshake)) with [scrubbed]: Invalid certificate chain!\n"

  [recv_certs/expired_rsa_id_ed25519 FAILED]

requested review from @nickm

This works for me on an M1 macbook pro: everything builds, and the tests pass (except as discussed in #40684 (closed)). I have a couple of questions, but they should not block merging.

---

My first question about this branch is the ways in which it differs from the rough draft in #40683 (closed):

• This script removes SYMDEF and SYMDEF SORTED, but the rough draft (which also worked for me) only removes SYMDEF SORTED.
• This script uses find to remove the offending directories wherever they appear. But the rough draft only removed SYMDEF SORTED at the top level.

Do we know if the approach in this script is necessary, or if we could "just" rm -rf '.__SYMDEF SORTED' ?

---

My second question is: Do we know why this happened? That is, why is this necessary? Why were these directories included in the .a files before, and why do they now prevent linking?

If there is a piece of documentation that we could link to that explains why this is happening, that would make me more confident that this approach is "correct" rather than "just happens to work".

approved this merge request

The initial rough draft was made to build ONLY on iOS - not macOS and the build system for the iOS build did what is equivalent to make libtor.a (and thus not building our test suites, including libtor-testing.a).

Since at least one of the archives in libtor-testing.a contained .__.SYMDEF (without SORTED) this issue would appear again when doing an ordinary make build (without a specific target) when our test binaries are linked against libtor-testing.a.

(please feel free to merge this whenever you like)

mentioned in merge request !636 (merged)

merged