application-services build on debian bookworm fails due to new python version
I'm trying to bring Tor Browser to F-Droid (tor-browser#27539 (comment 2989340)). F-Droid recently updated their build systems to Debian Bookworm. Since then, the previously working build recipe failed. First because you hard-code Java 11 here, which is fixed for now by installing Java 11 from the old debian repos (initial and fixup). (Switch to the default jdk? Does this break reproducibility?)
But the next problem breaks the application-services project. Log excerpt:
$ pip3 install glean_parser~=7.1.0
ERROR: Could not find a version that satisfies the requirement MarkupSafe<=2.0.1,>=1.1.1 (from glean-parser) (from versions: none)
ERROR: No matching distribution found for MarkupSafe<=2.0.1,>=1.1.1
After some time I got behind this issue. You disable internet access for pip (good). Instead, the wheels are fetched from a prebuilt tarball generated by your glean project. This tarball only contains python 3.9 prebuilt wheels for a couple of dependencies, among them MarkupSafe
. As Debian updated python from 3.9 to 3.11, those wheels cannot be used and the build fails.
Preserving compatibility in debian bookworm with python 3.9 currently doesn't work. Installing python 3.9 from the old repo works, but the python 3.9 venv package has some issues being installed on bookworm.
A proper solution might be to fetch source tarballs and build the wheels (at least those with arch-dependent components) locally during the build process. That would make it compatible with all current and future python versions. But it might lead to issues with reproducibility.
Another solution is obviously to update this container to debian bookworm and use python 3.11 consistently, here and at F-Droid.
Also consider the longer-term consequences. If Tor Browser were in F-Droid already we'd have problems updating it there. Either F-Droid provides a method to use an older version of their build VM system, or you have to ensure buildability with the next debian version once F-Droid updates their machines. This problem might get even more compicated if F-Droid uses reproducible builds (which is preferred and worked on bullseye except the signature copying). Because then you'd have to ensure reproducibility across different python (and other software, where applicable) versions (probably very hard to impossible) or upgrade to the new debian version yourself at about the same time. I guess this a big ask and F-Droid should provide a method to use older debian versions for build jobs.