Onion rewrites should add eTLDs

If we want to extend the possibility to add custom *.tor.onion (or some similar domains) in runtime, we should hack somehow nsEffectiveTLDService.

This feature is very important: it is used to split domains when scoping whatever needs to be domain-scoped (e.g., cookies). So, I expect it to be critical for FPI.

Actually, nsEffectiveTLDService already has a mechanism to update public suffixes, that we patch out (#40073 (closed)). But it uses some non trivial binary format that encodes a deterministic acyclic finite state automaton. This format is explained in xpcom/ds/tools/make_dafsa.py (the code itself is about ~200 Python rows).

If the only reason for #40073 (closed) to exist was our changes to the list, we could restore it, and find some way to inject our additional domains at every update of the automaton.

@JeremyRand was suggesting to use some prefs.

I also thought of adding some other methods to the IOnionAliasService interface that we implemented for #40458 (closed).

marked this issue as related to #40458 (closed)

marked this issue as related to #40073 (closed)

marked this issue as related to tor-browser-build#40518 (closed)

mentioned in issue tor-browser-build#40518 (closed)

The main criterion I'd have for this is that I'd like it to be possible for 3rd-party developers to add entries to the eTLD list (on a Tor Browser instance on a machine that they control) without needing Tor Project to take any action. At a minimum, that would mean not needing to rebuild Tor Browser from source. Exposing the API only via XPCOM would be undesirable since 3rd-party developers generally cannot install XPCOM extensions.

It would also be desirable to not require a WebExtension (because those have a Tivoization requirement that some 3rd-party developers may not consider acceptable). A pref would mostly work, except that it is nontrivial (AFAIK) for distro maintainers to set prefs in Tor Browser. Environment variables are trivial for distro maintainers to set. Enterprise policy rules are also (AFAIK) trivial for distro maintainers to set. I suspect that enterprise policy would be easier to eventually uplift to Mozilla than environment variables (it intuitively seems like an enterprisey feature, e.g. lots of enterprises have internal domains that they may wish to segregate via FPI), but I'm not very knowledgeable about what Mozilla's views are there. AFAIK Tor Browser disables all enterprise policy rules at the moment, so using enterprise policy for this might entail extra effort for Tor Browser.

I'm not particularly attached to any specific approach yet, so happy to discuss other ideas if you have any.

Exposing the API only via XPCOM would be undesirable since 3rd-party developers generally cannot install XPCOM extensions.

I see your point. Once we figure out a way to modify the automaton in runtime we could add more than one mechanism.

Having multiple ways to add eTLDs would satisfy everyone: the people with a static list of eTLDs, and us, if we modify the Onion alias patch to allow users create their namespaces (well, this is only an idea I had for a possible development that patch, but for the moment it isn't a real need, and we have not really discussed about it).

easier to eventually uplift to Mozilla

I didn't think to uplifting but you have a very solid point.

Once we create a way to add TLDs, we could also add a function to the original service, for us, rather than depending on our custom service, and see if we can get it uplifted with any other modification we may propose.

AFAIK Tor Browser disables all enterprise policy rules at the moment, so using enterprise policy for this might entail extra effort for Tor Browser.

Ehehe, we have a trick in our sleeve .

However, it may not be enough for our purposes.

I see your point. Once we figure out a way to modify the automaton in runtime we could add more than one mechanism.

Having multiple ways to add eTLDs would satisfy everyone: the people with a static list of eTLDs, and us, if we modify the Onion alias patch to allow users create their namespaces (well, this is only an idea I had for a possible development that patch, but for the moment it isn't a real need, and we have not really discussed about it).

If you're okay with having multiple ways to do it, then that sounds great to me. The highest-priority ways to do it from my point of view would be a pref and an enterprise policy; an environment variable would be desirable but somewhat lower priority; a WebExtensions API wouldn't really matter for me, but I imagine there may exist other developers who would like to have it.

Once we create a way to add TLDs, we could also add a function to the original service, for us, rather than depending on our custom service, and see if we can get it uplifted with any other modification we may propose.

Sounds good.

Ehehe, we have a trick in our sleeve

Oh cool! I didn't realize Tor Browser supported that trick. That'll be super helpful for some other development I'm doing (related to TLS for onion services) -- thanks for pointing me to that!

added Onion Services label

added Roadmap::Future label

added Feature label

added All Platforms label

mentioned in issue #40569

I've thought again about this problem earlier.

Actually, part of the code isn't even Moz's, but it's from Chromium.

So, I think this patch could be First Contribution because it doesn't involve knowing the inner workings of Firefox/Tor Browser patches.

I think we could use a port of the automaton encoder to a language Firefox can actually use in runtime, maybe JS with typed arrays?

At that point, if this script can also load the binary data, we could try to recreate the initial input data, add the domains we need, recreate the binary buffer and send it to C++ through XPCOM.

marked this issue as related to #41831 (closed)

mentioned in issue #41831 (closed)

Actually, I've just had another idea to solve this: we don't need to modify the automaton.

We could change the service to do some custom thing when the requested domain ends by .onion.

It if's a complete .onion, we could use the regular logic, otherwise, we could either force all aliases to be separate first parties (see my comments in #41831 (closed)), or do some other calls to other code.

@JeremyRand how do you rule subdomains in .bit.onion?

mentioned in issue #42616 (closed)

mentioned in issue #40789

added Apps::Type::Enhancement label

added Apps::Impact::Unknown label