When rustc generates its dependency file, it takes into account current
version of the codebase. If an update of the code then happens, with,
say an ipdl file being removed, the corresponding .h is no longer
generated and make rightfully complains about missing dependency.

Allow this kind of transition by providing a dummy rule for each
dependency generated by rustc.

Differential Revision: https://phabricator.services.mozilla.com/D178617

Instead of duplicating the flag location, use a single place to pass
either lto or lto=fat.

Differential Revision: https://phabricator.services.mozilla.com/D178841

When rustc generates its dependency file, it takes into account current
version of the codebase. If an update of the code then happens, with,
say an ipdl file being removed, the corresponding .h is no longer
generated and make rightfully complains about missing dependency.

Allow this kind of transition by providing a dummy rule for each
dependency generated by rustc.

Differential Revision: https://phabricator.services.mozilla.com/D178617

This combines the effect of full LTO for C/C++ code with rust cross LTO
capability.

Differential Revision: https://phabricator.services.mozilla.com/D172266

This combines the effect of full LTO for C/C++ code with rust cross LTO
capability.

Differential Revision: https://phabricator.services.mozilla.com/D172266

Differential Revision: https://phabricator.services.mozilla.com/D165808

Backed out changeset 8725bf468d29 (bug 1808184) for breaking incremental builds (bug 1826114). a=backout

Differential Revision: https://phabricator.services.mozilla.com/D165808

Looks like $(file <) construct is too recent for some of our downstream
users.

Differential Revision: https://phabricator.services.mozilla.com/D174210

Bug 1806618 - Avoid rebuilding RUST_PROGRAMS based on dependencies r=sylvestre,ahochheiden,firefox-build-system-reviewers

cargo generates dependency file for each program, systematically. Use it
to avoid redundant builds.

This handles the following situations:

* Cargo file is updated
* One of the dependency output by cargo is updated
* Binary is manually removed
* Dependency file is manually removed

Differential Revision: https://phabricator.services.mozilla.com/D165159

Backed out changeset adae02427fe6 (bug 1808184)
Backed out changeset 6445ca9d0ee4 (bug 1806618)

This generalizes Bug 1806618 to library files, avoiding a few extra calls
to rustc.

Differential Revision: https://phabricator.services.mozilla.com/D165808

Bug 1806618 - Avoid rebuilding RUST_PROGRAMS based on dependencies r=sylvestre,ahochheiden,firefox-build-system-reviewers

cargo generates dependency file for each program, systematically. Use it
to avoid redundant builds.

This handles the following situations:

* Cargo file is updated
* One of the dependency output by cargo is updated
* Binary is manually removed
* Dependency file is manually removed

Differential Revision: https://phabricator.services.mozilla.com/D165159

Backed out changeset 04b7be54155f (bug 1808184)
Backed out changeset e00332f32a2d (bug 1806618)

This generalizes Bug 1806618 to library files, avoiding a few extra calls
to rustc.

Differential Revision: https://phabricator.services.mozilla.com/D165808

Bug 1806618 - Avoid rebuilding RUST_PROGRAMS based on dependencies r=sylvestre,ahochheiden,firefox-build-system-reviewers

cargo generates dependency file for each program, systematically. Use it
to avoid redundant builds.

This handles the following situations:

* Cargo file is updated
* One of the dependency output by cargo is updated
* Binary is manually removed
* Dependency file is manually removed

Differential Revision: https://phabricator.services.mozilla.com/D165159

Bug 1810126 - Add a generic mechanism to run cargo-* commands. r=glandium,firefox-build-system-reviewers,ahochheiden

`mach cargo COMMAND` will run `cargo-COMMAND` using `cargo build`
arguments by default. However, it is possible to tune the arguments
using either command-line arguments, or a YAML file in `config/cargo/`.
A file `config/cargo/template.yaml` can be used to create a new
configuration for a new cargo sub-command.

Differential Revision: https://phabricator.services.mozilla.com/D166780

Differential Revision: https://phabricator.services.mozilla.com/D163956

Differential Revision: https://phabricator.services.mozilla.com/D163588

Bug 1802097 - Check return code of `cargo udeps` and `cargo audit` to determine if they aren't installed, and provide the necessary commands to install them if they aren't r=firefox-build-system-reviewers,glandium

Differential Revision: https://phabricator.services.mozilla.com/D162902

The patch is from Arthur Carcano <arthur.carcano@ocamlpro.com>, r=glandium

Differential Revision: https://phabricator.services.mozilla.com/D163551

Differential Revision: https://phabricator.services.mozilla.com/D162915

Depends on D160749

Differential Revision: https://phabricator.services.mozilla.com/D160750

Depends on D159838

Differential Revision: https://phabricator.services.mozilla.com/D160749

This adds a subcommand ./mach cargo udeps with the following
sub command arguments:

Sub Command Arguments:
  --all-crates          Check all of the crates in the tree.
  crates                The crate name(s) to check.
  --jobs [jobs], -j [jobs]
                        Run the tests in parallel using multiple
processes.
  -v, --verbose         Verbose output.
  --message-format-json
                        Emit error messages as JSON.
  --expect-unused       Do not return an error exit code if udeps
detects unused dependencies.

Differential Revision: https://phabricator.services.mozilla.com/D159838

Differential Revision: https://phabricator.services.mozilla.com/D161665

- The bug that was constantly disabling `VIRTUAL_TERMINAL_PROCESSING` has been fixed upstream in CYGWIN (https://github.com/cygwin/cygwin/commit/fdbd1539329ad669606767ab5a63a16f825b4c45) and merged into MSYS2 (https://github.com/msys2/msys2-runtime/commit/e4fdb0cb73cc48a2db473b525b6ba0a33ecabf97)
- The upcoming MozillaBuild `4.0.2` release will have the updated MSYS2 that contains this fix, allowing us to enable colored output on Windows.
- We will enable `ENABLE_VIRTUAL_TERMINAL_PROCESSING` once (For each stdout and stderr) on initializing the formatter to ensure that `VIRTUAL_TERMINAL_PROCESSING` is enabled whenever Mach is called..
- These changes will only be active if the MozillaBuild version is greater than or equal to Release `4.0.2`

Differential Revision: https://phabricator.services.mozilla.com/D138256

Backed out 7 changesets (bug 1744340, bug 1798827) for causing mach failures at test_commands.py. CLOSED TREE

Backed out changeset 9cb83a8d1f28 (bug 1798827)
Backed out changeset ebff36a14737 (bug 1744340)
Backed out changeset 5d6aabaf72c4 (bug 1744340)
Backed out changeset 8efafab5495f (bug 1744340)
Backed out changeset 8a873505f17d (bug 1744340)
Backed out changeset 3024fff68f6c (bug 1744340)
Backed out changeset 42a0a61583f0 (bug 1744340)

- The bug that was constantly disabling `VIRTUAL_TERMINAL_PROCESSING` has been fixed upstream in CYGWIN (https://github.com/cygwin/cygwin/commit/fdbd1539329ad669606767ab5a63a16f825b4c45) and merged into MSYS2 (https://github.com/msys2/msys2-runtime/commit/e4fdb0cb73cc48a2db473b525b6ba0a33ecabf97)
- The upcoming MozillaBuild `4.0.2` release will have the updated MSYS2 that contains this fix, allowing us to enable colored output on Windows.
- We will enable `ENABLE_VIRTUAL_TERMINAL_PROCESSING` once (For each stdout and stderr) on initializing the formatter to ensure that `VIRTUAL_TERMINAL_PROCESSING` is enabled whenever Mach is called..
- These changes will only be active if the MozillaBuild version is greater than or equal to Release `4.0.2`

Differential Revision: https://phabricator.services.mozilla.com/D138256

Bug 1791476 - When clang is newer than rust's llvm, use clang sanitizer runtimes. r=firefox-build-system-reviewers,ahochheiden

To do so, we move cargo-linker from shell to python because it's more
convenient.

Differential Revision: https://phabricator.services.mozilla.com/D161010

In order to allow rust-analyzer to be able to use the build script in
the mozbuild crate to discover the configuration information, this patch
adds new flags to the vscode config to tell rust-analyzer to invoke
cargo through `./mach cargo check`, and use the correct target directory
within the objdir rather than `$(topsrcdir)/target`.

Due to the virtual filesystem used by rust-analyzer not including files
in the object directory, this is not sufficient to get suggestions for
symbols from the included files, however it will accurately fetch
diagnostics upon save and run things like proc macros.

A new feature will likely need to be added to rust-analyzer to allow us
to specify additional paths to add to the source root for packages to
fix that issue.

Due to this change using `./mach cargo check`, rather than running it
independently, we don't run into issues caused by running `check`
against crates in the workspace which aren't being used, making the
diagnostics more useful.

An additional feature needed to be added to `./mach cargo check` to
allow specifying `--message-format=json`. I am open to suggestions for a
more elegant way to communicate this flag into the makefile.

Depends on D153269

Differential Revision: https://phabricator.services.mozilla.com/D153270

This switches the code to instead use the generated source file, which is
discovered based on `OUT_DIR` so is easier for rust-analyzer to understand.

The configuration for rust-analyzer will be updated in the next part to
make sure that rust-analyzer is able to take advantage of these changes,
and produce better diagnostics.

Differential Revision: https://phabricator.services.mozilla.com/D153269

Differential Revision: https://phabricator.services.mozilla.com/D88002

Differential Revision: https://phabricator.services.mozilla.com/D142617

As mentioned in bug 1747354, the location of the dist directory is
relied to be $topobjdir/dist, so just use that consistently rather
than getting it from a separate variable for rust build scripts.

Differential Revision: https://phabricator.services.mozilla.com/D136556

Backed out changeset 5ce212465a26 (bug 1751331)
Backed out changeset 14d62b8ffa5a (bug 1751331)
Backed out changeset 76f46bd9afa1 (bug 1751331)
Backed out changeset 147faaad046f (bug 1751331)
Backed out changeset f85b049d12f9 (bug 1751331)
Backed out changeset 16d8d3f8378b (bug 1751331)
Backed out changeset 1cd9386a3927 (bug 1751331)
Backed out changeset 20faacba6db6 (bug 1751331)
Backed out changeset 7d17f75bcb38 (bug 1751331)
Backed out changeset 28b9aab1f174 (bug 1751331)

As mentioned in bug 1747354, the location of the dist directory is
relied to be $topobjdir/dist, so just use that consistently rather
than getting it from a separate variable for rust build scripts.

Differential Revision: https://phabricator.services.mozilla.com/D136556

Differential Revision: https://phabricator.services.mozilla.com/D135219