Machine learning to let go

Merge Info

Issues

Resolves

• #44045 (closed)

Related

• #43867 (closed)
• #43562 (closed)
• #44009 (closed)

Merging

Target Branches

• tor-browser - !fixups to tor-browser-specific commits, new features, security backports
• base-browser and mullvad-browser - !fixups to base-browser-specific commits, new features to be shared with mullvad-browser, and security backports
  • All base-browser commits.

Target Channels

• Alpha: esr140-15.0
• Stable: esr128-14.5
• Legacy: esr115-13.5

Backporting

Timeline

• No Backport (preferred): patchset for the next major stable
• Immediate: patchset needed as soon as possible (fixes CVEs, 0-days, etc)
• Next Minor Stable Release: patchset that needs to be verified in nightly before backport
• Eventually: patchset that needs to be verified in alpha before backport

(Optional) Justification

• Security update: patchset contains a security fix (be sure to select the correct item in Timeline)
• Censorship event: patchset enables censorship circumvention
• Critical bug-fix: patchset fixes a bug in core-functionality
• Consistency: patchset which would make development easier if it were in both the alpha and release branches; developer tools, build system changes, etc
• Sponsor required: patchset required for sponsor
• Localization: typos and other localization changes that should be also in the release branch
• Other: please explain

Upstream

• Patchset is a candidate for uplift to Firefox
• Patchset is a backport from Firefox
  • Bugzilla link:
  • Upstream commit:

Issue Tracking

• Link resolved issues with appropriate Release Prep issue for changelog generation

Review

Request Reviewer

• Request review from an applications developer depending on modified system:
  • NOTE: if the MR modifies multiple areas, please /request_review all the relevant reviewers
  • accessibility : henry
  • android : clairehurst, dan
  • build system : boklm
  • ci/cd: brizental, henry
  • extensions : ma1
  • firefox internals (XUL/JS/XPCOM) : jwilde, ma1
  • fonts : pierov
  • frontend (implementation) : henry
  • frontend (review) : donuts, morgan
  • localization : henry, pierov
  • macOS : clairehurst, dan
  • nightly builds : boklm
  • rebases/release-prep : brizental, clairehurst, dan, ma1, pierov, morgan
  • security : jwilde, ma1
  • signing : boklm, morgan
  • updater : pierov
  • windows : jwilde, morgan
  • misc/other : pierov, morgan

Change Description

We switch off browser.ml.enabled, which disables MLEngine, which seems to be the backend for many, but not all, ml features.

We use preferences to switch off the following ML features:

1. Auto fill. Was only an experiment in 140, but has since been dropped upstream. Disabled by browser.ml.enable.
2. PDF auto fill of alt text. Disabled by pdfjs.enableAltText and pdfjs.enableGuessAltText.
3. Search suggests. Disabled by browser.urlbar.quicksuggest.mlEnabled (I think) and browser.ml.enable.
4. Smart tab grouping. Disabled by browser.tabs.groups.smart.enabled and browser.tabs.groups.smart.userEnabled.
5. Semantic history search. Disabled by browser.ml.enable and places.semanticHistory.featureGate.
6. Link preview. Disabled by browser.ml.linkPreview.enabled.
7. Chat sidebar. Disabled by browser.ml.chat.enable.
8. Exposure of ML engines for extensions. Disabled by extensions.ml.enabled and browser.ml.enable.

These preferences were also locked as well, since they can not work in the browser, and some would throw errors if the user tried to turn them on.

We also remove the "ml" and "genai" components entirely to remove the code that should be unused but is unaudited. We also exclude some further modules that are no longer viable without the "ml" or "genai" components.

And we tidy up a few places to either:

1. Remove the import of these missing files, and replace or comment out their consumers.
2. Or add a comment explaining why we think the missing file should never be loaded in practice due to our preferences.

We also lock down the translations component. I considered removing the entire component, but

1. The cld2 part is an established component. Needs a separate discussion.
2. TranslationsParent.sys.mjs has many callers, including from geckoview. I could have gone through and removed the desktop callers, but the android stuff is much more complicated. Could be done in the future if we think it is worth it.

Instead, we just target the engine parts that should be unused.

In addition, about:translations and about:inference are excluded. Technically about:inference is only in Firefox Nightly right now, but I wanted to future-proof its exclusion in case upstream is planning on bringing it to a non-Nightly release.

Finally, added a migration to clear all the locked preferences, so they do not linger in the user's profile.

Note on data clean up

By removing the files we are also removing the parts that control the stored ML data. As far as I could tell, all engines require fetching a wasm from remote settings, which will fail in Tor Browser. So I can't imagine many 15.0 alpha users have accumulated much engine data.

In contrast, the translations engine was already available in 14.0, but also needs to fetch a wasm with remote settings. So downloading languages, etc, would have failed even if the user switched browser.translations.enable in about:config.

Given this, I did not try and figure out how to clean out old ML data from the user's profile directory. But if someone can demonstrate lingering data, we can maybe try and do something about it.

Note on commit and comments

I placed a lot of tor-browser#44045 references in the comments. Even though I created a new commit "BB 44045: Disable ML features.", I imagine that we may have to make other issue-specific adjustments as part of this commit in the future, so I want to make it clear which parts came from the original issue, rather than future additions.

How Tested

No console errors in the following scenarios:

1. Browser startup.
2. New window.
3. Opening about:preferences.
4. Opening about:addons.
5. Grouping tabs.
6. Adding an image to a pdf.
7. Searching history.
8. Searching suggestions turned on in settings.
9. Opening sidebar.
10. Opening context menu.

Also checked that preferences in the locked preferences that appear in the user profile's prefs.js are cleared on migration.