Rebase Tor Browser patches for FF61

added TorBrowserTeam201806R component::applications/tor browser owner::arthuredelstein parent::26338 priority::very high resolution::fixed severity::normal sponsor::8 status::closed tbb-mobile type::enhancement labels

Trac:
Status: new to accepted
Owner: tbb-team to sysrqb

I was originally doing this as part of #25741 (moved), but it'll be easier for review if we factor this into its own child ticket.

Here's my branch, which I created by rebasing tor/tor-browser-60.0.1esr-8.0-1 (30544586f8a98e985e0038b2e905bfddb14e999e) onto a recent gecko-dev/beta (c432661a14b1ad331e22e2524ae8d57f9161884f):

https://github.com/arthuredelstein/tor-browser/commits/26233

I left out three updater patches that are complex to rebase and I think won't be needed for Android in any case (please correct me if I'm wrong!). I also squashed the fixup commits.

Here's what happened to each patch:

C = cherry-picked
F = fixed up
U = upstreamed
? = left out for now (updater patches)

C 30544586f8a9 fixup! TB4: Tor Browser's Firefox preference overrides.
F b135c59f65db Fix MAR generation bashism
C 1ba7b6b25113 fixup! TB4: Tor Browser's Firefox preference overrides.
C b589ec74c427 Bug 20283: Tor Browser should run without a `/proc` filesystem.
C 3a6cb718e815 Bug 21537: Tests for secure .onion cookies
C 58e4a739a6ed Bug 21537: Mark .onion cookies as secure
U 440d2fe1b6f4 Bug 1441327 - Allow for seccomp filtering of socket(AF_INET/AF_INET_6) calls on Linux when using UNIX Domain Sockets for SOCKS Proxy. r=bagder
C ed1a45a69d15 Bug 22548: Firefox downgrades VP9 videos to VP8.
C 550d0bae6d40 Bug 24398: Plugin-container process exhausts memory
F 3206814bc291 Bug 23104: Add a default line height compensation
C 53f7ab7d844a Bug 24052: Handle redirects by blocking them early
C 5e0170a7ca05 Bug 13398: at startup, browser gleans user FULL NAME (real name, given name) from O/S
C c5544f727e46 Bug 21830: Copying large text from web console leaks to /tmp
C 962babebfc5e Bug 21321: Add test for .onion whitelisting
F d18befdee332 Bug 21431: Clean-up system extensions shipped in Firefox 52
C fe68460a72cd Bug 21569: Add first-party domain to Permissions key
C 2aa950923c66 Bug 16285: Exclude ClearKey system for now
C 32da0487944c Bug 21907: Fix runtime error on CentOS 6
C d010f98a92fe Bug 21849: Don't allow SSL key logging
F d29c1ddb254d Bug #5741: Prevent WebSocket DNS leak.
F c67a0c07fd5b Bug 14970: Don't block our unsigned extensions
F 3549c5324dda Omnibox: Add DDG, Startpage, Disconnect, Youtube, Twitter;  remove Amazon, eBay, bing
C e984b8c54c75 Bug 23916: Add new MAR signing key
? 70989b5211d8 Bug 16940: After update, load local change notes.
? 189164e100db Bug 13379: Sign our MAR files.
? 4b4dcd40abae Bug 4234: Use the Firefox Update Process for Tor Browser.
C 7ca562c26856 Bug 25909: disable updater telemetry
C 39f10aaa10ef Bug 19121: reinstate the update.xml hash check
C 9a3bb35800d5 Bug 19121: reinstate the update.xml hash check
F a4ac08e62457 Bug 13252: Do not store data in the app bundle
C cab08be85615 Bug 21724: Make Firefox and Tor Browser distinct macOS apps
C 75d638dddd7d Bug 18912: add automated tests for updater cert pinning
C b95e30974e71 Bug 18900: updater doesn't work on Linux (cannot find  libraries)
C 716067b4c679 Bug 11641: change TBB directory structure to be more like Firefox's
C 4e0aed04f7f7 Bug 9173: Change the default Firefox profile directory to be TBB-relative.
C 71a812c584aa Bug 19890: Disable installation of system addons
C 17367581443f Bug 19273: Avoid JavaScript patching of the external app helper dialog.
C 4da1d08fb2e2 Bug 18923: Add a script to run all Tor Browser specific tests
C 612aefdabd9b Regression tests for #2874: Block Components.interfaces from content
C 27fa6ab6fa2b Bug 18821: Disable libmdns for Android and Desktop
C 40752ee655eb Bug 18800: Remove localhost DNS lookup in nsProfileLock.cpp
C cc9862c27fd5 Bug 18799: disable Network Tickler
C 5823d75f953a Bug 16620: Clear window.name when no referrer sent
C 48b1c08e1fff Bug 16441: Suppress "Reset Tor Browser" prompt.
C 6734d99f40b0 Bug 14392: Make about:tor behave like other initial pages.
C 006dffb468ee Bug 2176: Rebrand Firefox to TorBrowser
C 16a1bcef4e15 Bug 18995: Regression test to ensure CacheStorage is disabled in private browsing
C c5b57b1bf1df Regression tests for "Omnibox: Add DDG, Startpage, Disconnect, Youtube, Twitter; remove Amazon, eBay, bing"
C 6b35333f3a3a Regression tests for TB4: Tor Browser's Firefox preference overrides.
C 93a8e5d1b523 Regression tests for Bug #2950: Make Permissions Manager memory-only
C 1dc1a4f7fedc Bug 12620: TorBrowser regression tests folder
C b6d8bf568ba6 Bug 14631: Improve profile access error msgs (strings).
C 10ac5a7be31f Bug 14631: Improve profile access error messages.
C 87cb0833ffdd Bug 14716: HTTP Basic Authentication prompt only displayed once
C ccebcbb95267 Bug 13028: Prevent potential proxy bypass cases.
C dfd201a96767 Bug 16439: remove screencasting code.
F e0cb606a47ac Bug 2874: Block Components.interfaces from content
C 58a737f021b2 Bug 12974: Disable NTLM and Negotiate HTTP Auth
C fe1e6ce7f8d8 Bug 10280: Don't load any plugins into the address space.
C 837f8e888cf5 Bug 8312: Remove "This plugin is disabled" barrier.
C 1cbd34d3b0b8 Bug 3547: Block all plugins except flash.
F 0e8dbb37c450 TB4: Tor Browser's Firefox preference overrides.
C 4bdb543b0ae7 TB3: Tor Browser's official .mozconfigs.

Trac:
Status: accepted to needs_review

Trac:
Cc: sysrqb, igt0 to sysrqb, igt0, tbb-team
Keywords: TorBrowserTeam201805 deleted, TorBrowserTeam201806R added

Initial testing with rebasing this branch on top of my current #25741 (moved) branch is looking good! Thanks!

I added two fixups on top of my branch[0] that disable (ifdef-out) unused code which cause build failure when compiled with -Werror. This allows for running Try builds[1][2]. The HEAD commit is needed because it's possible mozilla-central and mozilla-beta don't have any commits by ffxbld within the last 500 commits (as is currently the situation).

I'm trying to get the Tor Browser mochitests working - but I won't spend too much time on this. I think I'm hitting a local issue.

[1] https://gitweb.torproject.org/user/sysrqb/tor-browser.git/log/?h=26233%2b25741 [1] https://treeherder.mozilla.org/#/jobs?repo=try&revision=aeeb247553c2d713478083d3f02bb9c9a78bf0a7 [2] https://treeherder.mozilla.org/#/jobs?repo=try&revision=c26bc84a8caffb2000475016edf5114b4e049193

Okay, here comes the review. I think MAR files and the signing machinery we have won't play a role for Android (see: #26242 (moved) for our update strategy), thus we can ignore all the signing related patches, I agree. I've been a bit more agressive when reviewing your changes. Here is all I got:

c7036c883efebaf0ee6d27285e7a5f9d0abe8eb8 -- good (4bdb543b0ae7) 2078afc6814a8f0303d9a83c050c068bda704ce3 -- not okay (0e8dbb37c450)

> +// Disable window.Components shim (substitue for https://bugs.torproject.org/2874)
> +pref("dom.use_components_shim", false);

"substitue" and could you mention the Mozilla bug number (1448048)? What's the relation to https://bugzilla.mozilla.org/1401260?

f2c4006d0958a61d671ad7ab8e4c097115ece39c -- okay (1cbd34d3b0b8) dfca44dbb3011918f1860ff10ddf7fa825b33713 -- okay (837f8e888cf5) 4df8383599da34d038e47980ce6005f6355d6a43 -- okay (fe1e6ce7f8d8) 5e9c8426b5d485ecc02c85cbb98d11305310ef79 -- okay (58a737f021b2)

F e0cb606a47ac Bug 2874: Block Components.interfaces from content <- I was confused by that because there is no bug 2874 related patch anymore. So, better "O" I guess.

5de7b7cdcf120b1fcf29f51d2ce2659d317ec445 -- okay (dfd201a96767) 9abd24dae812b2f3fc2918b2fb22285f9c3b1392 -- okay (ccebcbb95267) 8657933084abd6cb6745239698b8d24e11d69dc1 -- okay (87cb0833ffdd) ef0eee9cb5aa10e338c1979ff51d7fa6b90f6b0b -- okay (10ac5a7be31f) c96db008adde9b8f692786ce5dab817eca961507 -- okay (b6d8bf568ba6) ec4d2f41b75d3be3c239421fc6e6905d65efe8e7 -- okay (1dc1a4f7fedc) a8e5b7264ecf03e4d554d7a5cfead42159144d0d -- not okay (93a8e5d1b523)

I think that can go as the patch got upstreamed in bug 967812 (it seems I overlooked that by the review for ESR60). Or do we want to keep the test because the upstreamed patch does not contain tests? In that case I am fine with keeping it for now but we should get the test on the uplift radar in this case (why is it marked as tbb-no-uplift?).

174ce1dc9a00c7af8fd1019cc30e24903fada4d7 -- okay (6b35333f3a3a) 1a683a1d1d0079dafa15a4a3957da24182ea1f44 -- okay (c5b57b1bf1df) 36d44849da69cf008023519cb4bcede18e96c99c -- okay (16a1bcef4e15) 6da338503b55259c63160c5e24536ff1e77b5184 -- okay (006dffb468ee) 1b35f4478c904b0990a45d4a5f191fcc62f4b9ba -- okay (6734d99f40b0) 6300c93066d07ea3fa54c31852c1564992383877 -- okay (48b1c08e1fff) 184d59cff2b987c608d9746badd7a007f00423b8 -- okay (5823d75f953a) c4331c511caf1a2feaa95e6f6bfcbcd3393907b8 -- okay (cc9862c27fd5) f3140e62aaa16e05e9202d6dd4e656886bb285fd -- okay (40752ee655eb) e09d2897b4d67e3f65212269e6b774047bfe75fe -- okay (27fa6ab6fa2b) f1e3011eb042f516c24de3734b42a1e71d8744c1 -- not okay (612aefdabd9b)

It seems we don't need that test anymore given that the patch is basically obsolete (i.e. the fingerprinting defense got "upstreamed").

cee52946e9d5b16bb0c42d69e6896d7334995f58 -- okay (4da1d08fb2e2) ead92b734879ab153db67da293ac9cb6add8a186 -- okay (17367581443f) acef0857f3f6aeac13f4e94eb98f8b55d28e1127 -- okay (71a812c584aa) 79289c71dc16e3064e1ceb17fb1900e0d08273d8 -- not okay (4e0aed04f7f7)

Where are the changes in nsToolkitProfileService.cpp etc. coming from? I fail to find them on m-c. It seems to me we don't want to differe more than needed from Mozilla here.

f4c2dd434d4bb4aa5cbe4937c6f4e5f85eef74f9 -- okay (716067b4c679) 3f7ac7eb4d1fc0e66b3105fa105c639aea134733 -- not okay (b95e30974e71)