Skip to content
Snippets Groups Projects
Commit c58d0e6b authored by Mitchell Hentges's avatar Mitchell Hentges
Browse files

Bug 1730712: Command virtualenvs should include Mach's import scope. r=ahal

Mach's import scope includes:
* Its `pth` entries
* Its pip packages, which is either:
    * The Mach virtualenv site-packages, if packages were "pip
      installed" over the internet.
    * The system environment's site-packages, if installing packages
      over the internet is disabled and Mach is grabbing packages from
      the system Python instead.

Command virtualenvs _already_ had this import scope when they were
dynamically activated from an existing Mach process. However, when
used directly (e.g. by `<venv>/bin/python <script>`), they would be
missing this import scope, which was a confusing inconsistency.

However, resolving this inconsistency adds a new risk: when Mach is
using the system Python, the system packages now populate the same
context as command virtualenv packages - and they hadn't been checked
for compatibility. So, this patch also
includes behaviour to verify system<=>command-venv compatibility at
activation-time.

A few notes about this system-package-verification:
* It happens at virtualenv activation-time instead of build-time because
  system packages may change after the virtualenv is built.
* It takes roughly 1.5s, which is significant, but it should mainly occur
  in CI, where it's acceptable. Devs using `MACH_USE_SYSTEM_PACKAGES`
  should unset the variable to avoid the time delay.
* The algorithm works by asserting top-level requirements (e.g.
  `psutil>=5.4.2,<=5.8.0`), then runs `pip check` over the combined set
  of packages that would be in-scope.
* Note that, in this patch, system packages are *not* asserted to be
  the same version as their vendored counterparts. This is because, until
  we parse `third_party/python/requirements.in`, we don't know which
  packages we depend on directly (and whose APIs we care about if they
  change). Since leaning on system packages is essentially only used in
  CI (which we have strong control on), this downside seemed acceptable.

Differential Revision: https://phabricator.services.mozilla.com/D126288
parent cea54423
No related branches found
No related tags found
No related merge requests found
Loading
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment