-
- Downloads
Bug 1730712: Command virtualenvs should include Mach's import scope. r=ahal
Mach's import scope includes: * Its `pth` entries * Its pip packages, which is either: * The Mach virtualenv site-packages, if packages were "pip installed" over the internet. * The system environment's site-packages, if installing packages over the internet is disabled and Mach is grabbing packages from the system Python instead. Command virtualenvs _already_ had this import scope when they were dynamically activated from an existing Mach process. However, when used directly (e.g. by `<venv>/bin/python <script>`), they would be missing this import scope, which was a confusing inconsistency. However, resolving this inconsistency adds a new risk: when Mach is using the system Python, the system packages now populate the same context as command virtualenv packages - and they hadn't been checked for compatibility. So, this patch also includes behaviour to verify system<=>command-venv compatibility at activation-time. A few notes about this system-package-verification: * It happens at virtualenv activation-time instead of build-time because system packages may change after the virtualenv is built. * It takes roughly 1.5s, which is significant, but it should mainly occur in CI, where it's acceptable. Devs using `MACH_USE_SYSTEM_PACKAGES` should unset the variable to avoid the time delay. * The algorithm works by asserting top-level requirements (e.g. `psutil>=5.4.2,<=5.8.0`), then runs `pip check` over the combined set of packages that would be in-scope. * Note that, in this patch, system packages are *not* asserted to be the same version as their vendored counterparts. This is because, until we parse `third_party/python/requirements.in`, we don't know which packages we depend on directly (and whose APIs we care about if they change). Since leaning on system packages is essentially only used in CI (which we have strong control on), this downside seemed acceptable. Differential Revision: https://phabricator.services.mozilla.com/D126288
Showing
- configure.py 26 additions, 3 deletionsconfigure.py
- python/mach/mach/site.py 467 additions, 105 deletionspython/mach/mach/site.py
- python/mach/mach/test/test_decorators.py 23 additions, 4 deletionspython/mach/mach/test/test_decorators.py
- python/mozbuild/mozbuild/base.py 4 additions, 2 deletionspython/mozbuild/mozbuild/base.py
- python/mozbuild/mozbuild/controller/building.py 10 additions, 9 deletionspython/mozbuild/mozbuild/controller/building.py
- python/mozbuild/mozbuild/mach_commands.py 0 additions, 1 deletionpython/mozbuild/mozbuild/mach_commands.py
- python/mozbuild/mozbuild/sphinx.py 4 additions, 2 deletionspython/mozbuild/mozbuild/sphinx.py
- python/mozbuild/mozbuild/test/test_vendor.py 17 additions, 13 deletionspython/mozbuild/mozbuild/test/test_vendor.py
Loading
Please register or sign in to comment