Skip to content

Bug 42247: Add some Android facilities to use TorProvider also there

Pier Angelo Vendrame requested to merge pierov/tor-browser:bug_42247 into tor-browser-115.5.0esr-13.5-1

Merge Info

Related Issues

Backporting

Timeline

  • Immediate: patchset needed as soon as possible
  • Next Minor Stable Release: patchset that needs to be verified in nightly before backport
  • Eventually: patchset that needs to be verified in alpha before backport
  • No Backport (preferred): patchset for the next major stable

(Optional) Justification

  • Emergency security update: patchset fixes CVEs, 0-days, etc
  • Censorship event: patchset enables censorship circumvention
  • Critical bug-fix: patchset fixes a bug in core-functionality
  • Consistency: patchset which would make development easier if it were in both the alpha and release branches; developer tools, build system changes, etc
  • Sponsor required: patchset required for sponsor
  • Other: please explain

Merging

  • Merge to tor-browser - !fixups to tor-browser-specific commits, new features, security backports
  • Merge to base-browser - !fixups to base-browser-specific commits, new features to be shared with mullvad-browser, and security backports
    • NOTE: if your changeset includes patches to both base-browser and tor-browser please clearly label in the change description which commits should be cherry-picked to base-browser after merging

Issue Tracking

Review

Request Reviewer

  • Request review from an applications developer depending on modified system:

Change Description

So, I was discussing on IRC about these tasks, and Dan told me there were some issues assigned to him.

However, in my exploration from before the hackweek, I had already touched many parts.

I think they could be reviewed already and be used as a starting point for Dan (we could even merge them, they'll be active only in nighly builds).

I haven't written Java/Android seriously for a long time, so my skill might be quite bad now 😅.

Java-JS (and C++) plumbing in a nutshell

The main way to plumb data between languages seems to be the EventDispatcher.

So, we have event-driven architecture, similar to what we already do with Service.obs. It works well with our current codebase, since we use async/await a lot (callbacks can be easily transformed into async in JS).

As an approximation, it allows to plumb JSON object up and down. We see them as normal objects on the JS side, and as GeckoBundle in Java. A GeckoBundle is a string-keyed map. Values can be string, integers, doubles, other GeckoBundles, or array of them.

Process handles

The most arguable part of the patch is that the processes are more or less could be singletons, but we don't do anything for enforcing that at the moment.

Dealing with that as long as it's only JS is easy: we have the references to these processes (and we just need to remember to kill them when they're not needed anymore - at least for lyrebird, we never kill tor, but we take its ownership, so it commits suicide when the owning control port connection disconnects).

From the Java side it's less easy, because I think there's some serialization involved that loses the GC information.

So, I decided to allow to create multiple processes at once, and to put them into a hashmap. For lyrebird, it seemed to me that it was fine to create an handle on the Java side, so I used a counter, which was the easiest way to do it.

For Tor it's more involved because we delete objects and creating the handle on the JS side might help us to deal with possible race conditions. Also, I've modeled the two things in a different way, see in the next session.

Expected lifetime

I'd expect lyrebird processes to have a short lifetime: we need to do some domain fronting only at the bootstrap. Even in bad cases, the domain fronting should end within a few minutes (e.g., 10 minute bootstrap in China 😱).

So, having a process in the Gecko runtime makes sense to me. In addition to that, we have a thread to read the stdout and find the domain-fronting proxy credentials, instead of blocking everything (everything is async anyway).

For tor it might be a whole different story. Right now we have the same implementation of lyrebird, and it worked well... in the emulator.

But we might want to change. For this reason, the events I implemented for tor are different than for the meek transport, and they are:

  • Java has started tor, JS can proceed with control port connection and so on
  • Java failed to start tor, JS should go through the TorProvider initialization failure branch
  • tor has exited: the code is ready to see this even before the other events! That's why I generated the handle on the JS side

This should allow to switch to an Android service, and we can create a broadcast receiver or whatever to deal with the conversion between native Android mechanisms and EventDispatcher or whatever.

I tried to go through Android docs, but I think it should be better if an actual Android developer dealt with that.

How is it integrated with GeckoView?

For the domain isolator, I found a file that from my tests is ran only once, which is what we want.

Some other files are run for each tab (GeckoSession).

So, since we have a file we can use, I made it initialize also TorProviderBuilder, TorSettings and TorConnect. But only when we're not in release/alpha (so, nightly and "default", which is the channel for the dev builds).

The Java facilites are there, but at the moment they don't do anything, except for registering to events. They start doing things only when JS asks them to.

What now with the domain isolator?

I was curious to see if it could gather the data for the circuit display on Andorid... It could, but it didn't gather it, because on Android we don't have tabs/multiple browsers, only one browser.

But that property is available also on desktop, but I think it's the browser that is currently visible, which might be different from the one that started the request (I think they're always the same on Android).

However, if they have the same browsingContext, we don't have to go through all the browsers also on desktop, and we can shortcircuit it.

This change was totally unnecessary for this MR, but it teased me, so I just did it. I also have some changes not included in this branch to see if it was possible to plumb the data down for finally realizing the circuit display on Android (yes, they can, but it's also harder than it should be, each tab keeps a state, rather than querying the session, so everything is just harder).

Cleaning on shutdown

I create a random temporary directory to store the IPC data on it. We do more or less the same on desktop (to possibly handle old sessions improperly closed, or race conditions on the same paths when restarting the tor daemon).

I couldn't find a hook to call the shutdown function and clear these old data. This is something we should do before merging, but I'll need some help to find a nice place to do it.

PTs, torrc-default and GeoIP

As of creating the MR, we're still missing a few files, or we have them but we don't pass them to tor.

It's easy to solve if we already have the file, but I had already asked Dan to check this branch (well, the same branch under the torconnect-toolkit name), so I wanted to create a MR first, to keep a trace of all the additional changes.

Speaking of having the needed files: this MR takes for granted it can find the binaries in lib/$abi or whatever (we never use the architecture explicitly, we ask Android the correct path). It works for now that we still include tor-android-service and tor-onion-proxy-library, but we'll have to do something about that (e.g., tor-browser-build#41010).

Slightly related: in addition to include the PTs, we need to include a torrc-default to use them. I think we could use relative paths, and start tor in ..../lib/$abi (again, or whatever that is). Right now we don't have a file, we do something else in Java. As a temporary solution I could include a file in Fenix, but I think we should include it when we deal with tor-browser-build#41010.

Guardian project's (updated) tor-service/tor-android-service

That implements part of this MR. I'm not sure what it does works (well) for us, and we're not interested in its control port implementation.

Also, it depends on GP's tor fork that adds JNI bindings.

So, switching to it instead will require some additional work.

connectionWorked, #didConnectToTorControlPort on TorProcess

I've implemented a TorProcessAndroid that looks similar to TorProcess (basically, same interface).

I wrote a comment about these functions being useless after some changes.

So, instead of conditionally calling it only on desktop/implementing it empty on Android, I decided just to finally remove it.

Work for Fenix

This needs the following in Fenix:

  1. disable the current bootstrap procedure in nightlies
  2. possibly make about:torconnect fullscreen
  3. something else I forgot 😒

How Tested

Tor bootstrap

Local build.

Firefox-android needs to be patched to skip the bootstrap:

diff --git a/fenix/app/src/main/java/org/mozilla/fenix/HomeActivity.kt b/fenix/app/src/main/java/org/mozilla/fenix/HomeActivity.kt
index 9396abc907..8cff48e4a1 100644
--- a/fenix/app/src/main/java/org/mozilla/fenix/HomeActivity.kt
+++ b/fenix/app/src/main/java/org/mozilla/fenix/HomeActivity.kt
@@ -1163,7 +1163,7 @@ open class HomeActivity : LocaleAwareAppCompatActivity(), NavHostActivity {
     }
 
     open fun navigateToHome() {
-        navHost.navController.navigate(NavGraphDirections.actionStartupTorbootstrap())
+        navHost.navController.navigate(NavGraphDirections.actionStartupOnboarding())
     }
 
     override fun attachBaseContext(base: Context) {

After that, proxy requests will fail.

We need to manually go to about:torconnect and bootstrap from there.

Domain fronting

I set torbrowser.debug.censorship_level to 1. It makes the bootstrap fail and prompt your location. The select is populated automatically with the data we get from the domain front.

Circuit display changes (bonus!)

For the circuit display collection, I checked through console logs (browser.tordomainisolator.loglevel set to all, USB debugging enabled, and the console loaded from about:debugging in Firefox).

Edited by Pier Angelo Vendrame

Merge request reports