Whenever a valid Onion-Location HTTP header (or corresponding HTML
<meta> http-equiv attribute) is found in a document load, we either
redirect to it (if the user opted-in via preference) or notify the
presence of an onionsite alternative with a badge in the urlbar.

A custom HTTPS Everywhere update channel is installed,
which provides rules for locally redirecting some memorable
.tor.onion URLs to non-memorable .onion URLs.

When these redirects occur, we also rewrite the URL in the urlbar
to display the human-memorable hostname instead of the actual
.onion.

Bug 34196: Update site info URL with the onion name

Bug 40456: Update the SecureDrop HTTPS-Everywhere update channel
Bug 40478: Onion alias url rewrite is broken

When Tor informs the browser that client authentication is needed,
temporarily load about:blank instead of about:neterror and prompt
for the user's key.

If a correctly formatted key is entered, use Tor's ONION_CLIENT_AUTH_ADD
control port command to add the key (via Torbutton's control port
module) and reload the page.

If the user cancels the prompt, display the standard about:neterror
"Unable to connect" page. This requires a small change to
browser/actors/NetErrorChild.jsm to account for the fact that the
docShell no longer has the failedChannel information. The failedChannel
is used to extract TLS-related error info, which is not applicable
in the case of a canceled .onion authentication prompt.

Add a leaveOpen option to PopupNotifications.show so we can display
error messages within the popup notification doorhanger without
closing the prompt.

Add support for onion services strings to the TorStrings module.

Add support for Tor extended SOCKS errors (Tor proposal 304) to the
socket transport and SOCKS layers. Improved display of all of these
errors will be implemented as part of bug 30025.

Also fixes bug 19757:
 Add a "Remember this key" checkbox to the client auth prompt.

 Add an "Onion Services Authentication" section within the
 about:preferences "Privacy & Security section" to allow
 viewing and removal of v3 onion client auth keys that have
 been stored on disk.

Also fixes bug 19251: use enhanced error pages for onion service errors.

Encrypting pages hosted on Onion Services with SSL/TLS is redundant
(in terms of hiding content) as all traffic within the Tor network is
already fully encrypted.  Therefore, serving HTTP pages from an Onion
Service is more or less fine.

Prior to this patch, Tor Browser would mostly treat pages delivered
via Onion Services as well as pages delivered in the ordinary fashion
over the internet in the same way.  This created some inconsistencies
in behaviour and misinformation presented to the user relating to the
security of pages delivered via Onion Services:

 - HTTP Onion Service pages did not have any 'lock' icon indicating
   the site was secure
 - HTTP Onion Service pages would be marked as unencrypted in the Page
   Info screen
 - Mixed-mode content restrictions did not apply to HTTP Onion Service
   pages embedding Non-Onion HTTP content

This patch fixes the above issues, and also adds several new 'Onion'
icons to the mix to indicate all of the various permutations of Onion
Services hosted HTTP or HTTPS pages with HTTP or HTTPS content.

Strings for Onion Service Page Info page are pulled from Torbutton's
localization strings.

eBay and Amazon don't treat Tor users very well. Accounts often get locked and
payments reversed.

Also:
Bug 16322: Update DuckDuckGo search engine

We are replacing the clearnet URL with an onion service one (thanks to a
patch by a cypherpunk) and are removing the duplicated DDG search
engine. Duplicating DDG happend due to bug 1061736 where Mozilla
included DDG itself into Firefox. Interestingly, this caused breaking
the DDG search if JavaScript is disabled as the Mozilla engine, which
gets loaded earlier, does not use the html version of the search page.
Moreover, the Mozilla engine tracked where the users were searching from
by adding a respective parameter to the search query. We got rid of that
feature as well.

Also:
This fixes bug 20809: the DuckDuckGo team has changed its server-side
code in a way that lets users with JavaScript enabled use the default
landing page while those without JavaScript available get redirected
directly to the non-JS page. We adapt the search engine URLs
accordingly.

Also fixes bug 29798 by making sure we only specify the Google search
engine we actually ship an .xml file for.

Also regression tests.

squash! Omnibox: Add DDG, Startpage, Disconnect, Youtube, Twitter; remove Amazon, eBay, bing

Bug 40494: Update Startpage search provider

squash! Omnibox: Add DDG, Startpage, Disconnect, Youtube, Twitter; remove Amazon, eBay, bing

Bug 40438: Add Blockchair as a search engine

Bug 33342: Avoid disconnect search addon error after removal.

We removed the addon in #32767, but it was still being loaded
from addonStartup.json.lz4 and throwing an error on startup
because its resource: location is not available anymore.

It's time for our rotation again: Move the backup key in the front
position and add a new backup key.

Bug 33803: Move our primary nightly MAR signing key to tor-browser

Bug 33803: Add a secondary nightly MAR signing key

Add an about:tbupdate page that displays the first section from
TorBrowser/Docs/ChangeLog.txt and includes a link to the remote
post-update page (typically our blog entry for the release).

Always load about:tbupdate in a content process, but implement the
code that reads the file system (changelog) in the chrome process
for compatibility with future sandboxing efforts.

Also fix bug 29440. Now about:tbupdate is styled as a fairly simple
changelog page that is designed to be displayed via a link that is on
about:tor.

Configure with --enable-verify-mar (when updating, require a valid
  signature on the MAR file before it is applied).
Use the Tor Browser version instead of the Firefox version inside the
  MAR file info block (necessary to prevent downgrade attacks).
Use NSS on all platforms for checking MAR signatures (instead of using
  OS-native APIs, which Mozilla does on Mac OS and Windows). So that the
  NSS and NSPR libraries the updater depends on can be found at runtime,
  we add the firefox directory to the shared library search path on macOS.
  On Linux, rpath is used by Mozilla to solve that problem, but that
  approach won't work on macOS because the updater executable is copied
  during the update process to a location that is under TorBrowser-Data,
  and the location of TorBrowser-Data varies.

Also includes the fix for bug 18900.

Bug 19121: reinstate the update.xml hash check

Revert most changes from Mozilla Bug 1373267 "Remove hashFunction and
hashValue attributes from nsIUpdatePatch and code related to these
attributes." Changes to the tests were not reverted; the tests have
been changed significantly and we do not run automated updater tests
for Tor Browser at this time.

Also partial revert of commit f1241db6.

Revert the nsUpdateService.js changes from Mozilla Bug 862173 "don't
verify mar file hash when using mar signing to verify the mar file
(lessens main thread I/O)."

Changes to the tests were not reverted; the tests have been changed
significantly and we do not run automated updater tests for
Tor Browser at this time.

We kept the addition to the AppConstants API in case other JS code
references it in the future.

The following files are never updated:
  TorBrowser/Data/Browser/profiles.ini
  TorBrowser/Data/Browser/profile.default/bookmarks.html
  TorBrowser/Data/Tor/torrc
Mac OS: Store update metadata under TorBrowser/UpdateInfo.
Removed the %OS_VERSION% component from the update URL (13047) and
  added support for minSupportedOSVersion, an attribute of the
  <update> element that may be used to trigger Firefox's
  "unsupported platform" behavior.
Hide the "What's new" links (set app.releaseNotesURL value to about:blank).
Windows: disable "runas" code path in updater (15201).
Windows: avoid writing to the registry (16236).
Also includes fixes for tickets 13047, 13301, 13356, 13594, 15406,
  16014, 16909, 24476, and 25909.

Also fix Bug 26049: reduce the delay before the update prompt is displayed.
Instead of Firefox's 2 days, we use 1 hour (after which time the update
doorhanger will be displayed).

Also fix bug 27221: purge the startup cache if the Tor Browser
version changed (even if the Firefox version and build ID did
not change), e.g., after a minor Tor Browser update.

Also fix 32616: Disable GetSecureOutputDirectoryPath() functionality.

Bug 26048: potentially confusing "restart to update" message

Within the update doorhanger, remove the misleading message that mentions
that windows will be restored after an update is applied, and replace the
"Restart and Restore" button label with an existing
"Restart to update Tor Browser" string.

Bug 28885: notify users that update is downloading

Add a "Downloading Tor Browser update" item which appears in the
hamburger (app) menu while the update service is downloading a MAR
file. Before this change, the browser did not indicate to the user
that an update was in progress, which is especially confusing in
Tor Browser because downloads often take some time. If the user
clicks on the new menu item, the about dialog is opened to allow
the user to see download progress.

As part of this fix, the update service was changed to always show
update-related messages in the hamburger menu, even if the update
was started in the foreground via the about dialog or via the
"Check for Tor Browser Update" toolbar menu item. This change is
consistent with the Tor Browser goal of making sure users are
informed about the update process.

Removed #28885 parts of this patch which have been uplifted to Firefox.

Also added 'New circuit for this site' button to CustomizableUI, but
not visible by default.

This patch adds a new 'securitylevel' component to Tor Browser intended
to replace the torbutton 'Security Slider'.

This component adds a new Security Level toolbar button which visually
indicates the current global security level via icon (as defined by the
extensions.torbutton.security_slider pref), a drop-down hanger with a
short description of the current security level, and a new section in
the about:preferences#privacy page where users can change their current
security level. In addition, the hanger and the preferences page will
show a visual warning when the user has modified prefs associated with
the security level and provide a one-click 'Restore Defaults' button to
get the user back on recommended settings.

Strings used by this patch are pulled from the torbutton extension, but
en-US defaults are provided if there is an error loading from the
extension. With this patch applied, the usual work-flow of "./mach build
&& ./mach run" work as expected, even if the torbutton extension is
disabled.

This loads HTTPS Everywhere as a builtin addon from a hardcoded
resource:// URI in desktop. It also ensures that the non-builtin
HTTPS Everywhere addon is always uninstalled on browser startup.

The reason of making this desktop-only is that there are some issues
when installing a builtin extension from geckoview side, making
the extension not available on first startup. So, at least for
now we handle the Fenix case separately. See #40118 for a followup
for investigating these.

Regression tests for Bug #2950: Make Permissions Manager memory-only

Regression tests for TB4: Tor Browser's Firefox preference overrides.

Note: many more functional tests could be made here

Regression tests for #2874: Block Components.interfaces from content

Bug 18923: Add a script to run all Tor Browser specific tests

Regression tests for Bug #16441: Suppress "Reset Tor Browser" prompt.

- implements new about:torconnect page as tor-launcher replacement
- adds tor connection status to url bar and tweaks UX when not online
- adds new torconnect component to browser
- tor process management functionality remains implemented in tor-launcher through the TorProtocolService module
- adds warning/error box to about:preferences#tor when not connected to tor
- explicitly allows about:torconnect URIs to ignore Resist Fingerprinting (RFP)
- various tweaks to info-pages.inc.css for about:torconnect (also affects other firefox info pages)

Bug 40773: Update the about:torconnect frontend page to match additional UI flows

This patch adds a new about:preferences#connection page which allows
modifying bridge, proxy, and firewall settings from within Tor Browser.
All of the functionality present in tor-launcher's Network
Configuration panel is present:

 - Setting built-in bridges
 - Requesting bridges from BridgeDB via moat
 - Using user-provided bridges
 - Configuring SOCKS4, SOCKS5, and HTTP/HTTPS proxies
 - Setting firewall ports
 - Viewing and Copying Tor's logs
 - The Networking Settings in General preferences has been removed

Bug 40774: Update about:preferences page to match new UI designs

When handling an external URI or downloading a file, invoke Torbutton's
external app blocker component (which will present a download warning
dialog unless the user has checked the "Automatically download files
from now on" box).

For e10s compatibility, avoid using a modal dialog and instead use
a callback interface (nsIHelperAppWarningLauncher) to allow Torbutton
to indicate the user's desire to cancel or continue each request.

Other bugs fixed:
 Bug 21766: Crash with e10s enabled while trying to download a file
 Bug 21886: Download is stalled in non-e10s mode
 Bug 22471: Downloading files via the PDF viewer download button is broken
 Bug 22472: Fix FTP downloads when external helper app dialog is shown
 Bug 22610: Avoid crashes when canceling external helper app downloads
 Bug 22618: Downloading pdf file via file:/// is stalling

Adds a CryptoSafety actor which detects when you've copied a crypto
address from a HTTP webpage and shows a warning.

Closes #40209.

Bug 40428: Fix string attribute names

Instead of always reporting that the profile is locked, display specific
messages for "access denied" and "read-only file system".

To allow for localization, get profile-related error strings from Torbutton.
Use app display name ("Tor Browser") in profile-related error alerts.

See Bug 1357997 for partial uplift.

Also:
Bug 28051 - Use our Orbot for proxying our connections

Bug 31144 - ESR68 Network Code Review

Build and package Tor Launcher as part of the browser (similar to
how pdfjs is handled).

If a Tor Launcher extension is present in the user's profile, it is
removed.

Because of the non-restartless nature of Torbutton, it required
a two-stage installation process. On mobile, it was a problem,
because it was not loading when the user opened the browser for
the first time.

Moving it to tor-browser and making it a system extension allows it
to load when the user opens the browser for first time.

Additionally, this patch also fixes Bug 27611.

Bug 26321: New Circuit and New Identity menu items

Bug 14392: Make about:tor behave like other initial pages.

Bug 25013: Add torbutton as a tor-browser submodule

Bug 31575: Replace Firefox Home (newtab) with about:tor

Avoid loading AboutNewTab in BrowserGlue.jsm in order
to avoid several network requests that we do not need. Besides,
about:newtab will now point to about:blank or about:tor (depending
on browser.newtabpage.enabled) and about:home will point to
about:tor.

- migrated in-page settings read/write implementation from about:preferences#tor
  to the TorSettings module
- TorSettings initially loads settings from the tor daemon, and saves them to
  firefox prefs
- TorSettings notifies observers when a setting has changed; currently only
  QuickStart notification is implemented for parity with previous preference
  notify logic in about:torconnect and about:preferences#tor
- about:preferences#tor, and about:torconnect now read and write settings
  thorugh the TorSettings module
- all tor settings live in the torbrowser.settings.* preference branch
- removed unused pref modify permission for about:torconnect content page from
  AsyncPrefs.jsm

Bug 40645: Migrate Moat APIs to Moat.jsm module

Before reordering patches, we used to keep the Tor-related patches
(torbutton and tor-launcher) at the beginning.
After that issue, we decided to move them towards the end, however we
kept TB4: Tor Browser's Firefox preference overrides at the beginning
because it influcences many other features.
As a result, to keep bisect working, we split that commit, and moved
all the preferences related to Tor (such as network.proxy.*) here.

Also:
Bug #9829.1: new .mozconfig file for the new cross-compiler and ESR24
Changes needed to build Mac in 64bit
Bug 10715: Enable Webgl for mingw-w64 again.
Disable ICU when cross-compiling; clean-up.
Bug 15773: Enable ICU on OS X
Bug 15990: Don't build the sandbox with mingw-w64
Bug 12761: Switch to ESR 38 for OS X
Updating .mozconfig-asan
Bug 12516: Compile hardenend Tor Browser with -fwrapv
Bug 18331: Switch to Mozilla's toolchain for building Tor Browser for OS X
Bug 17858: Cannot create incremental MARs for hardened builds.
Define HOST_CFLAGS, etc. to avoid compiling programs such as mbsdiff
(which is part of mar-tools and is not distributed to end-users) with
ASan.
Bug 13419: Add back ICU for Windows
Bug 21239: Use GTK2 for ESR52 Linux builds
Bug 23025: Add hardening flags for macOS
Bug 24478: Enable debug assertions and tests in our ASan builds
--enable-proxy-bypass-protection
Bug 27597: ASan build option in tor-browser-build is broken

Bug 27623 - Export MOZILLA_OFFICIAL during desktop builds

This fixes a problem where some preferences had the wrong default value.
Also see bug 27472 where we made a similar fix for Android.

Bug 30463: Explicitly disable MOZ_TELEMETRY_REPORTING

Bug 31450: Set proper BINDGEN_CFLAGS for ASan builds

Add an --enable-tor-browser-data-outside-app-dir configure option

Add --with-tor-browser-version configure option

Bug 21849: Don't allow SSL key logging.

Bug 31457: disable per-installation profiles

The dedicated profiles (per-installation) feature does not interact
well with our bundled profiles on Linux and Windows, and it also causes
multiple profiles to be created on macOS under TorBrowser-Data.

Bug 31935: Disable profile downgrade protection.

Since Tor Browser does not support more than one profile, disable
the prompt and associated code that offers to create one when a
version downgrade situation is detected.

Bug 32493: Disable MOZ_SERVICES_HEALTHREPORT

Bug 25741 - TBA: Disable features at compile-time

MOZ_NATIVE_DEVICES for casting and the media player
MOZ_TELEMETRY_REPORTING for telemetry
MOZ_DATA_REPORTING for all data reporting preferences (crashreport, telemetry, geo)

Bug 25741 - TBA: Add default configure options in dedicated file

Define MOZ_ANDROID_NETWORK_STATE and MOZ_ANDROID_LOCATION

Bug 29859: Disable HLS support for now

Add --disable-tor-launcher build option

Add --enable-tor-browser-update build option

Bug 33734: Set MOZ_NORMANDY to False

Bug 33851: Omit Parental Controls.

Bug 40061: Omit the Windows default browser agent from the build

Bug 40107: Adapt .mozconfig-asan for ESR 78

Bug 40252: Add --enable-rust-simd to our tor-browser mozconfig files

Bug 40793: moved Tor configuration options from old-configure.in to moz.configure

Reuse the Firefox onboarding mechanism with minimal changes.
Localizable strings are pulled in from Torbutton (if Torbutton is
  not installed, we lack about:tor and no tour will be shown).
Replace SVG images with PNGs (see bug 27002),
For defense in depth, omit include OnboardingTelemetry.jsm entirely.
Added support for the following UITour page event:
  torBrowserOpenSecuritySettings

Also fix bug 27403: the onboarding bubble is not always displayed.

Arthur suggested to make the onboarding bubble visible on displays with
less than 960px width available, so we choose 200px instead.

Also fix bug 28628: Change onboarding Security panel to open new
Security Level panel.

Also fix bug 27484: Improve navigation within onboarding.

Bug 27082: enable a limited UITour

Disallow access to UITour functionality from all pages other than
  about:home, about:newtab, and about:tor.
Implement a whitelist mechanism for page actions.

Bug 26962 - implement new features onboarding (part 1).

Add an "Explore" button to the "Circuit Display" panel within new user
onboarding which opens the DuckDuckGo .onion and then guides users through
a short circuit display tutorial.

Allow a few additional UITour actions while limiting as much as possible
how it can be used.

Tweak the UITour styles to match the Tor Browser branding.

All user interface strings are retrieved from Torbutton's
browserOnboarding.properties file.

Bug 27486 Avoid about:blank tabs when opening onboarding pages.

Instead of using a simple <a href>, programmatically open onboarding
web pages by using tabBrowser.addTab(). The same technique is now
used for "See My Path", "See FAQs", and "Visit an Onion".

Bug 29768: Introduce new features to users

Add an "update" tour for the Tor Browser 8.5 release that contains two
panels: Toolbar and Security (with appropriate description text and
images).

Display an attention-grabbing dot on the onboarding text bubble when
the update tour is active. The animation lasts for 14 seconds.

Bug 31768: Introduce toolbar and network settings changes in onboarding

Update the "Tor Network" onboarding page to include a note that
settings can now be accessed via the application preferences and
add an "Adjust Your Tor Network Settings" action button which opens
about:preferences#tor.

Replace the Tor Browser 8.5 "update" onboarding tour with a 9.0 one
that includes the revised "Tor Network" page and a revised "Toolbar"
page. The latter explains that Torbutton's toolbar item has been
removed ("Goodbye Onion Button") and explains how to access the
New Identity feature using the hamburger menu and new toolbar item.

Bug 34321 - Add Learn More onboarding item

Bug 40429: Update Onboarding for 10.5

Revert "Bug 1462415 - Delete onboarding system add-on r=Standard8,k88hudson"

This reverts commit f7ffd78b.

Revert "Bug 1498378 - Actually remove the old onboarding add-on's prefs r=Gijs"

This reverts commit 057fe36f.

Bug 28822: Convert onboarding to webextension

Partially revert 1564367 (controlCenter in UITour.jsm)

See also Bugs #5194, #7187, #8115, #8219.

This patch does some basic renaming of Firefox to TorBrowser. The rest of the
branding is done by images and icons.

Also fix bug 27905.

Bug 25702: Update Tor Browser icon to follow design guidelines

- Updated all of the branding in /browser/branding/official with new 'stable'
icon series.
- Updated /extensions/onboarding/content/img/tor-watermark.png with new icon and
add the source svg in the same directory
- Copied /browser/branding/official over /browser/branding/nightly and the new
/browser/branding/alpha directories. Replaced content with 'nightly' and
'alpha' icon series.
Updated VisualElements_70.png and VisualElements_150.png with updated icons in
each branding directory (fixes #22654)
- Updated firefox.VisualElementsManfiest.xml with updated colors in each
branding directory
- Added firefox.svg to each branding directory from which all the other icons
are derived (apart from document.icns and document.ico)
- Added default256.png and default512.png icons
- Updated aboutTBUpdate.css to point to branding-aware icon128.png and removed
original icon
- Use the Tor Browser icon within devtools/client/themes/images/.

Bug 30631: Blurry Tor Browser icon on macOS app switcher

It would seem the png2icns tool does not generate correct icns files and
so on macOS the larger icons were missing resulting in blurry icons in
the OS chrome. Regenerated the padded icons in a macOS VM using
iconutil.

Bug 28196: preparations for using torbutton tor-browser-brand.ftl

A small change to Fluent FileSource class is required so that we
can register a new source without its supported locales being
counted as available locales for the browser.

Bug 31803: Replaced about:debugging logo with flat version

Bug 21724: Make Firefox and Tor Browser distinct macOS apps

When macOS opens a document or selects a default browser, it sometimes
uses the CFBundleSignature. Changing from the Firefox MOZB signature to
a different signature TORB allows macOS to distinguish between Firefox
and Tor Browser.

Bug 32092: Fix Tor Browser Support link in preferences

For bug 40562, we moved onionPattern* from bug 27476 to here, as
about:tor needs these files but it is included earlier.

CSS and JS changes to alter the UX surrounding letterboxing. The
browser element containing page content is now anchored to the bottom
of the toolbar, and the remaining letterbox margin is the same color
as the firefox chrome. The letterbox margin and border are tied to
the currently selected theme.

Also adds a 'needsLetterbox' property to tabbrowser.xml to fix a race
condition present when using the 'isEmpty' property. Using 'isEmpty'
as a proxy for 'needsLetterbox' resulted in over-zealous/unnecessary
letterboxing of about:blank tabs.

Only use regional OS locales if the pref
`intl.regional_prefs.use_os_locales` is set to true.

Many fonts have issues with their vertical metrics. they
are used to influence the height of ascenders and depth
of descenders. Gecko uses it to calculate the line height
(font height + ascender + descender), however because of
that idiosyncratic behavior across multiple operating
systems, it can be used to identify the user's OS.

The solution proposed in the patch uses a default factor
to be multiplied with the font size, simulating the concept
of ascender and descender. This way all operating
systems will have the same line height only and only if the
frame is outside the chrome.

Patch written by Neill Miller

Convert JS implementation (within Torbutton) to a C++ browser patch.

Unless the -osint command line flag is used, the browser now defaults
to the equivalent of -no-remote.  There is a new -allow-remote flag that
may be used to restore the original (Firefox-like) default behavior.

It looks like these cases should only be invoked in the NSS command line
tools, and not the browser, but I decided to patch them anyway because there
literally is a maze of network function pointers being passed around, and it's
very hard to tell if some random code might not pass in the proper proxied
versions of the networking code here by accident.

Restrict the Enterprise Policies mechanism to only consult a
policies.json file (avoiding the Windows Registry and macOS's
file system attributes).

Add a few disabledByPolicy() checks to the update service to
avoid extraneous (and potentially confusing) log messages when
updates are disabled by policy.

Sample content for distribution/policies.json:
{
  "policies": {
    "DisableAppUpdate": true
  }
}

On Linux, avoid reading policies from /etc/firefox/policies/policies.json