Create a README.md file

# Firefox Application Services

# Application Services

Application Services (a-s) is a collection of Rust Components that are used to enable Firefox applications to integrate with Firefox accounts, sync, experimentation, etc. Each component is built using a core of shared code written in Rust, wrapped with native language bindings for different platforms.

This repository contains the fork of application-services used by Tor Browser for Android.

### Contributing

To contribute, please review the Mozilla [Community Participation Guidelines](https://www.mozilla.org/en-US/about/governance/policies/participation/) and then visit our [how to contribute](docs/contributing.md) guide.

The main differences from this project to the original are a few patches that allow

it to be built in tor-browser-build and patches that disable  most of the features

from application-services either run time or build time.

### Contact

Get in touch with other community members on Matrix, or through issues here on GitHub.

- Matrix: [#rust-components:mozilla.org](https://chat.mozilla.org/#/room/#rust-components:mozilla.org) ([How to connect](https://wiki.mozilla.org/Matrix#Connect_to_Matrix))

## Structure

# Documentation

> **tl;dr;** It's a bunch of Rust components grouped together in the "megazord"

> crate and exposed to Android code through auto-generated Kotlin bindings.

### High-level docs

The bulk of the logic implemented in application-services is written in Rust and

separated into components. All components are in the `/components` directory.

The [Application Services Book](https://mozilla.github.io/application-services/book/index.html) contains high-level documentation about the code in this repository.  It's built from the [./docs/](docs) directory.

This components are gathered in the "megazord" Rust crate, to be conveniently

added into application as a single library. There are multiple "megazord" crates

in `/megazord`, but the only one we care about is the one under `megazords/full`.

### Package docs

The features exposed by the megazord have to be accessed from Android applications --

and iOS, but for us that is not supported and thus irrelevant. For that, Kotlin bindings

are generated using Uniffi when building each component.

We use rustdoc to document both the public API of the components and the various internal implementation details.  View them on [https://mozilla.github.io/application-services/book/rust-docs/fxa_client/index.html](https://mozilla.github.io/application-services/book/rust-docs/fxa_client/index.html).  Once you have completed the build steps, you can view the docs by running:

## Building

```shell

cargo doc --no-deps --document-private-items --open

The process to build this project is similar to the original build steps of

application-services with a few tweaks.

> **Note**: For detailed information on the original steps, check out application-services

> own documentation at the [Application Services Book](https://mozilla.github.io/application-services/book/index.html).

>

> Also, you might want to check the original README in the upstream repository,

> or have a look at [upstream's build instructions](docs/building.md).

### Setup

#### uniffi-rs

Before building application-services it's necessary to procure a build of our fork of

uniffi-rs. There are two possibilities for that, either you build it yourself:

```sh

git clone https://gitlab.torproject.org/tpo/applications/uniffi-rs.git

cd uniffi-rs

cargo build [--release]

```

Or you can grab a build from https://tb-build-06.torproject.org/~tb-builder/tor-browser-build/out/uniffi-rs/

#### Toolchain setup

This project requires specific versions of the Android toolchain to be available. The exact versions of each component can be checked in `settings.gradle` (search for `ndkVersion`).

##### Android SDK

After all the required components are downloaded, it is recommended to set the location of the Android SDK in `local.properties`, like so:

```properties

sdk.dir=/path/to/android-sdk

```

This is recommended because it will always prefer this path, even if building from a different folder with different toolchain defaults -- which is the case when substituting a-s for a local build of tor-browser.

> **Note**: Usually application-services uses the same SDK and NDK versions as required by tor-browser. When using `./mach bootstrap` to download toolchains in tor-browser, you can set `sdk.dir` to `$HOME/.mozbuild/android-sdk-linux`:

>

> ```properties

> sdk.dir=/home/<you>/.mozbuild/android-sdk-linux

> ```

##### Android NDK

The build expects the `ndk` to be located inside `/path/to/android-sdk/ndk/XX.X.XXXXXX` (where `XX.X.XXXXXX` is the version found in `settings.gradle` for the NDK).

You can easily find it with this command:

```sh

grep ndkVersion settings.gradle

```

# Building

If you are using Firefox's SDK and NDK, you will need to create a symlink to it, as the NDK is not _inside_ the SDK folder, but _next_ to it:

### Building the Rust Components

1. Clone or Download the repository:

```shell

  $ git clone https://github.com/mozilla/application-services # (or use the ssh link)

  $ cd application-services

  $ git submodule init

  $ git submodule update --recursive

```sh

mkdir -p ~/.mozbuild/android-sdk-linux/ndk

cd ~/.mozbuild/android-sdk-linux/ndk

ln -s ../../android-ndk-rXYZ XX.X.XXXXXX

```

2. Follow these instructions to install your [system-level dependencies](docs/building.md#building-application-services)

3. Run the a-s Rust unit tests

```shell

cargo test

##### Rust

Make sure all the Android Rust targets are available, before building:

```sh

rustup target add aarch64-linux-android

rustup target add x86_64-linux-android

rustup target add armv7-linux-androideabi

rustup target add i686-linux-android

```

### Consumer build, integration and testing

The application-services library primary consumers are Fenix (Firefox on Android) and Firefox iOS. Assure you are able to run integration tests (for Android and iOS if using MacOS) by following the instructions to build for Android and iOS integrations.  

#### Android integration builds and helpful tools

* Build instructions to test [Fenix / android-components integration](docs/building.md#building-for-fenix)

* [Fenix Auto-publication workflow for android-components and application-services](https://github.com/mozilla-mobile/fenix/#auto-publication-workflow-for-android-components-and-application-services)

#### Firefox for iOS integration

* Build instructions to test [Firefox iOS integration](docs/building.md#building-for-firefox-ios)

#### Firefox Desktop

* Build instructions to test [Firefox Desktop integration](docs/building.md#building-for-firefox-desktop)

# Rust Components

[./components/](components) contains the source for each component. Note that most components have their FFI generated

by the [uniffi](https://github.com/mozilla/uniffi-rs/) library.

* See [./components/places/](components/places) for an example, where you can

    find:

  * The shared [rust code](components/places/src).

  * The [Kotlin bindings](components/places/android) for use by Android

      applications.

  * The [Swift bindings](components/places/ios) for use by iOS applications.

* See [./components/fxa-client](components/fxa-client) for an example that uses

    [uniffi](https://github.com/mozilla/uniffi-rs/) to generate API wrappers for

    multiple languages, such as Kotlin and Swift.

### List of components

* [ads-client](/components/ads-client) - for fetching ads via UAPI

* [autofill](components/autofill) - for storage and syncing of credit card and

  address information

* [crashtest](components/crashtest) - testing-purposes (crashing the Rust code)

* [fxa-client](components/fxa-client) - for applications that need to sign in

  with FxA, access encryption keys for sync, and more.

* [logins](components/logins) - for storage and syncing of a user's saved login

  credentials

* [nimbus](components/nimbus) - for integrating with Mozilla's [experimentation](https://mozilla.github.io/experimenter-docs/) platform for Firefox

* [places](components/places) - for storage and syncing of a user's saved

  browsing history

* [push](components/push) - for applications to receive real-time updates via

  WebPush

* [remote-settings](components/remote-settings) - for integrating with [Remote Settings](https://remote-settings.readthedocs.io/)

* [support](components/support) - low-level utility libraries

  * [support/rc_crypto](components/support/rc_crypto) - handles cryptographic needs backed by Mozilla's

    [NSS](https://developer.mozilla.org/en-US/docs/Mozilla/Projects/NSS) library

  * [support/sql](components/support/sql) - utilities for storing data locally

    with SQL

* [sync15](components/sync15) - shared library for accessing data in Firefox

  Sync

* [sync_manager](components/sync_manager) - integrates multiple sync engines/

  stores into a single framework

* [tabs](components/tabs) - an in-memory syncing engine for remote browser tabs

* [viaduct](components/viaduct) - an HTTP request library

* [webext-storage](components/webext-storage) - powers an implementation of the

chrome.storage.sync WebExtension API

Note the above list is partial; see the actual list under the `components` directory.

##### gyp

For some reason, NSS uses the `gyp` project generator rather than some more common tool such as CMake.

On Linux, you should find it in your distribution:

```sh

# Debian, Ubuntu, and derivatives

apt install gyp

# Arch Linux and derivatives

pacman -Sy gyp

```

On macOS, you can install `gyp-next` from `pip`:

```sh

python3 -m pip install gyp-next

```

`gyp` in turn uses `ninja`, but it is very likely you already have it in your system.

### Build

The first part of the build happens in the `libs` directory:

```sh

cd libs

./build-all.sh desktop

```

This builds both some dependencies (such as NSS) and the unpatched megazord.

We need to do it, as the no-op version grabs the interfaces to create from the unpatched one.

After that, we need to build NSS for Android:

```

export ANDROID_HOME=$HOME/.mozbuild/android-sdk-linux/

export ANDROID_NDK_HOME=$HOME/.mozbuild/android-ndk-r29/

export ANDROID_NDK_ROOT=$ANDROID_NDK_HOME

./build-all.sh android

```

And then the Android bindings:

```sh

cd .. # Assuming you were in the libs directory, otherwise go to the repo root.

./gradlew -PuniffiBindgenNoop="/path/to/the/uniffi-bindgen/binary" assemble(Release|Debug)

```

The uniffi binary is usually `.../uniffi-rs/target/release/bindgen`.

> **Note**: the shell won't escape `~`, so be sure not to use it to specify this path. Instead you can use `$HOME`.

> **Note**: This command can fail when calling `linker-wrapper.sh` which is hardcoded to call `python`. If you are in an env with only `python3` it will fail. You can append `env RUST_ANDROID_GRADLE_PYTHON_COMMAND=python3` to the front of this command to tell it to use `python3`

If you want to package it:

```sh

./gradlew -PuniffiBindgenNoop="/path/to/the/uniffi-bindgen/binary" publish

```

That will create a new local maven repository in `build/maven`.

You can merge it to your existing local repo with this command:

```sh

cp -R build/maven/org ~/.m2/repository/

```

## Troubleshooting

### Cleanup

Just running `clean` in `gradle` is not enough, as it will not delete the binaries built with Cargo.

There isn't an official cleanup script for the `libs` directory.

So, in doubt, you might run `rm -rf android desktop`.

It might be useful also to remove the various `target` directories (one is in the root of the repository, the other one is `megazords/full/target`).

After the cleanup, be sure to re-run also the steps in the `libs` directory!

### Failed to open file `.../libmegazord.so`

Re-run `./build-all.sh desktop` in the `libs` directory.