fix builds for Android
This does fix builds for both the arti binary and the tests. Most tests seem to be passing, some are failing. I will try to investigate them and send fixes/create issue to highlight them. Hopefully this helps arti in being production ready fast.
Merge request reports
Activity
requested review from @wesleyac
@CodingThunder Thanks for this patch!
What does your build toolchain look like? This looks pretty reasonable, but I don't have any recent experience building Rust on Android, and I'd like to verify that this works (and see how much effort it would take to add Android to our CI so we could catch this kind of thing in the future).
In particular:
- What commands are you using to build it?
- Are you running the tests, or just compiling them?
- Are you running this on a emulator, physical device, or something else?
CC @cve who I think has experience working on Android stuff for onionmasq and might have more context to review this :)
This work started as an attempt to port arti to Termux (an open source terminal emulator for Androiid). You can see the build scripts which I use to build arti here: https://github.com/termux/termux-packages/pull/21993. There are a lot of other things happening which is not in the script which you can find in the same GitHub repository under the ./scripts directory. For the build environment, I did the build on two different environments:
- Termux with Rust 1.84.1 on an Android device
Basically just install rust and rust using the package manager on Termux, and git clone the repo and run
cargo test --no-fail-fast.- On a docker image with Android NDK in path
This is the environment which we use at https://github.com/termux/termux-packages. You can find the Dockerfile used to create the container here: https://github.com/termux/termux-packages/blob/master/scripts/Dockerfile. I can give you a small summary of how it works. rust and cargo are installed using rustup. Android NDK from Google is in path. Now it's just a matter of building with
cargo build --target <android-target-name>. I have done the builds for 4 architectures (arm64/aarch64, arm, i686/x86, x86_64).Regarding the testing, I did run the tests on Termux after building it on Termux itself. Most tests just seem to pass, only a few are failing, which I will be looking into later once I find the time for it. It should also be possible to build the tests on a Linux machine using
cargo build --tests, then those tests can be transferred to a physical device for running the tests. I just prefer to build it on device as it's not that slow since this project isn't that large and I can avoid the hassle of transferring the built binaries. I am not a Rust guy, so I am not sure how much possible this is.Hopefully this answers all of your questions, in I missed something or you need further clarification feel free to point out.
And for the adding Android to the CI, I think I might be able to help with that (atleast for the build part). For automated testing on CI, I haven't done it. I have used https://github.com/termux/termux-docker with some scripts locally for testing nodejs locally for Termux, but I don't think this is going to be a solution here. The test results aren't consistent with actual test runs on a physical device as docker shares the same kernel as the host with the container, and Android devices have very different build of Linux kernel, with often OEMs adding their own salt to the mixture. Maybe running the tests inside a VM would be a better thing to do for something like Arti. Also the termux-docker repository does contain some binary blobs extracted from Android images.
-
Hmmm, it would be great to have if you want to help!
I think cross-compiling to android in CI would be a good start, and if we want to run tests, doing that in a Android emulator would probably be the way to go (although that might be fairly heavy for our CI machines, we'd need to look into that)
Here is a quick patch that adds an Android build target to CI for various architectures and all features.
diff --git a/.gitlab-ci.yml b/.gitlab-ci.yml index c8ea4394c..924977222 100644 --- a/.gitlab-ci.yml +++ b/.gitlab-ci.yml @@ -295,6 +295,21 @@ rust-recent-async-std-rustls: - rustup component add clippy - cd crates/arti-client && cargo clippy --no-default-features --features=async-std,rustls +rust-android-all-features: + stage: build + image: containers.torproject.org/tpo/core/onionmasq:latest + script: + - rustup show + - rustup target install armv7-linux-androideabi + - rustup target install aarch64-linux-android + - rustup target install i686-linux-android + - rustup target install x86_64-linux-android + - cargo install cargo-ndk + - cargo ndk --target armv7-linux-androideabi build --all-features + - cargo ndk --target aarch64-linux-android build --all-features + - cargo ndk --target i686-linux-android build --all-features + - cargo ndk --target x86_64-linux-android build --all-features + rust-clippy-nontest: stage: test image: $RECENT_RUST_IMAGEAt the moment, it fails of course, due to the errors this MR tries to fix.1
I am not sure if its better to do that in a separate merge requests?
@micah Right now, this uses an onionmasq container, is it somehow possible to get an Android container here as well?
Edited by Clara Engler
@clara - yes! The base-images project provides such images. Right now, it doesn't have an Android container, but it would be nice to add one.
I am not sure if its better to do that in a separate merge requests?
Do you want the CI to be done in the same MR? I think it's better to have a different MR. Also this MR is ready to be merged if the answer is no.
The CI config which you shared seems good enough, but I think you might be interested in building for different target architectures parallely to speed up the build. Or else it's going to take a long time compared to builds for linux, macos and windows
Edited by Yaksh Bariya
mentioned in commit aa7c0e0a
