tor-browser-build issueshttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues2023-11-01T20:14:46Zhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/41000Automation2023-11-01T20:14:46ZrichardAutomation*The* Automation ~Meta Ticket tracking our myriad automation efforts
High level summary:
- Release Prep Automation
- basically convert as much of the checklist to an automatic'ish walk-through script
- Enable GeckoDriver everywhere...*The* Automation ~Meta Ticket tracking our myriad automation efforts
High level summary:
- Release Prep Automation
- basically convert as much of the checklist to an automatic'ish walk-through script
- Enable GeckoDriver everywhere
- CI/Automation
- Builds
- Linting
- Auto-rebasing
- Tests
- tor-browser/mullvad-browser specific tests
- TZP-based fingerprinting
- Fix+Run Mozilla Tests
- Builing Signing Improvements
- per build signing
- Mullvad Buildinghttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40661Revert projects/mingw-w64-clang/mingw-windows_foundation.patch after upstream...2022-12-22T10:52:06ZrichardRevert projects/mingw-w64-clang/mingw-windows_foundation.patch after upstream widl issue is fixedUpstream: https://bugs.winehq.org/show_bug.cgi?id=53431Upstream: https://bugs.winehq.org/show_bug.cgi?id=53431https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40644How to build on fedora OS2023-12-06T09:41:19ZhenryHow to build on fedora OS## Problem
If I try and build on fedora OS with
```
make torbrowser-nightly-linux-x86_64
```
it fails to complete
```
...
Building project browser - tor-browser-tbb-nightly.2022.10.18-linux-x86_64-7f0811
Building project container-im...## Problem
If I try and build on fedora OS with
```
make torbrowser-nightly-linux-x86_64
```
it fails to complete
```
...
Building project browser - tor-browser-tbb-nightly.2022.10.18-linux-x86_64-7f0811
Building project container-image - container-image_jessie-amd64-59db6374f8fc.tar.gz
Building project mmdebstrap-image - container-image_jessie-amd64-1.tar.gz
Using file [...]/tor-browser-build/out/mmdebstrap/mmdebstrap-src-0.8.6-dff0a4.tar.gz
Using file [...]/tor-browser-build/out/mmdebstrap-image/container-image_ubuntu-base-22.04-base-amd64.tar.gz
Build log: [...]/tor-browser-build/logs/mmdebstrap-image.log
Error running build
```
Looking in `mmdebstrap-image.log` the first error I see is
```
dpkg: error processing archive /var/cache/apt/archives//acl_2.2.52-2_amd64.deb (--install):
cannot get security labeling handle: No such file or directory
```
followed by lots of messages like this:
```
dpkg: regarding .../base-files_8+deb8u11_amd64.deb containing base-files, pre-dependency problem:
base-files pre-depends on awk
awk is not installed.
```
## Expected
Some way to build on fedora OS. We should add this information to `README` or`/doc/BUILD_ERRORS.txt`Sponsor 131 - Phase 5 - Ongoing Maintenanceboklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40542Split up rbm.conf2023-08-26T06:05:20ZGeorg KoppenSplit up rbm.confThe options in `rbm.conf` are either `tor-browser-build` specific or should move to `rbm` or be module-dependent. We should go over them and split things out accordingly.The options in `rbm.conf` are either `tor-browser-build` specific or should move to `rbm` or be module-dependent. We should go over them and split things out accordingly.Sponsor 131 - Phase 5 - Ongoing Maintenanceboklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40098Clean up macosx-toolchain config file2023-01-05T12:52:02ZGeorg KoppenClean up macosx-toolchain config fileWe should figure out whether `CC` etc. still needs to get set in the
config file of the `macosx-toolchain` project. Chances are high that
this can get cleaned up given that a lot of projects get the necessary
info from `rbm.conf`'s `conf...We should figure out whether `CC` etc. still needs to get set in the
config file of the `macosx-toolchain` project. Chances are high that
this can get cleaned up given that a lot of projects get the necessary
info from `rbm.conf`'s `configure_opt` and for Firefox we have its
mozconfig file.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40056Get all mobile dependencies during dry run2023-07-17T12:52:26ZGeorg KoppenGet all mobile dependencies during dry runThe first build after a version bump is usually with networking allowed
to get all the Gradle dependencies extracted. That step is currently
failing for `application-services`, `android-components`, and `fenix` for unknown reasons: the G...The first build after a version bump is usually with networking allowed
to get all the Gradle dependencies extracted. That step is currently
failing for `application-services`, `android-components`, and `fenix` for unknown reasons: the Gradle
`--debug` output does not show artifact downloads for some dependencies
for later re-use in our mavenLocal setup.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40047Build our GCC properly2023-06-01T16:35:24ZGeorg KoppenBuild our GCC properlyGCC depends on MPC which depends on MPFR which depends on GMP.
Please, update GCC with dependencies everywhere.
(That was originally the purpose of #31845).
See Mozilla's GCC build process for all the deps. We might not need all
of the...GCC depends on MPC which depends on MPFR which depends on GMP.
Please, update GCC with dependencies everywhere.
(That was originally the purpose of #31845).
See Mozilla's GCC build process for all the deps. We might not need all
of them, though.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40014Improve lucetc/wasi-sdk integration2023-05-02T13:56:01ZGeorg KoppenImprove lucetc/wasi-sdk integrationWe need to leave some rough edges to our lucetc/wasi-sdk integration in order to get nightly builds based on ESR 78 going as soon as possible. They need to get fixed for the final toolchain, though.
- [x] #33488
- [ ] #40043
- [x] #4004...We need to leave some rough edges to our lucetc/wasi-sdk integration in order to get nightly builds based on ESR 78 going as soon as possible. They need to get fixed for the final toolchain, though.
- [x] #33488
- [ ] #40043
- [x] #40044
- [x] #40049
- [x] #40050
- [ ] #40063https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/34434Remove unused $var: 1 declarations in rbm.conf2023-01-05T13:53:36ZGeorg KoppenRemove unused $var: 1 declarations in rbm.confThere are a bunch of unused $var: 1 declarations in `rbm.conf` (that is they don't have a matching `[% c("var/$var") %]` check). We should clean that up and keep things tidy so that it is obvious where things need to get added and where ...There are a bunch of unused $var: 1 declarations in `rbm.conf` (that is they don't have a matching `[% c("var/$var") %]` check). We should clean that up and keep things tidy so that it is obvious where things need to get added and where they are not needed.Sponsor 131 - Phase 5 - Ongoing Maintenancehttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/34301Fix shellcheck issues in our tor-browser-build scripts2023-01-05T15:03:10ZGeorg KoppenFix shellcheck issues in our tor-browser-build scriptsWe add more and more shell scripts for different tasks into our `tor-browser-build` repo, which is great. We should go over the already existing ones and fix `shellcheck` issues.
This is the parent ticket for that effort.We add more and more shell scripts for different tasks into our `tor-browser-build` repo, which is great. We should go over the already existing ones and fix `shellcheck` issues.
This is the parent ticket for that effort.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/34203Some of the static libraries we build are not reproducible2023-01-05T14:34:24ZGeorg KoppenSome of the static libraries we build are not reproducibleI just realized that the `.a` archives we create (e.g.) for `libevent` on android are not reproducible while their contents are. We should fix that as it makes it easier to compare results and spot problems.
While we are at it we should...I just realized that the `.a` archives we create (e.g.) for `libevent` on android are not reproducible while their contents are. We should fix that as it makes it easier to compare results and spot problems.
While we are at it we should check other outputs as well as I bet not only `lilbevent` is affected.
FWIW: In the `libevent` case it seems timestamps play a role when creating the `.a` files.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/34110Investigate `./mach android gradle-dependencies` for our use cases2022-08-03T12:18:56ZGeorg KoppenInvestigate `./mach android gradle-dependencies` for our use casesMozilla has a neat way of automating the gradle dependencies it needs during build time and making them available: https://firefox-source-docs.mozilla.org/build/buildsystem/toolchains.html#firefox-for-android-with-gradle
We should think...Mozilla has a neat way of automating the gradle dependencies it needs during build time and making them available: https://firefox-source-docs.mozilla.org/build/buildsystem/toolchains.html#firefox-for-android-with-gradle
We should think about how we could use that either just for Fenix or in general for our mobile related projects.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/34022Rename the testbuild/torbrowser-testbuild targets2023-01-05T12:55:57ZboklmRename the testbuild/torbrowser-testbuild targetsFor release, alpha and nightly we use the same target name in `projects/release/config` and other components.
For testbuild however, we use the `testbuild` target in `projects/release/config`, which is using the `torbrowser-testbuild` t...For release, alpha and nightly we use the same target name in `projects/release/config` and other components.
For testbuild however, we use the `testbuild` target in `projects/release/config`, which is using the `torbrowser-testbuild` target (defined in rbm.conf) for building other components, which can be confusing. There is also a `testbuild` target in rbm.conf, which is used by `torbrowser-testbuild`.
I think we could fix that by renaming the `torbrowser-testbuild` target we have in `rbm.conf` to `testbuild`, and the current `testbuild` target to `testbuild-common`.
This will however require that people who use a custom `rbm.local.conf` update it.Sponsor 131 - Phase 5 - Ongoing Maintenancehttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40563Start using a maintained version of osslsigncode for our authenticode signing2023-01-05T12:46:01ZGeorg KoppenStart using a maintained version of osslsigncode for our authenticode signing`osslsigncode` on SoureForge seems to be dead for a while now. It's worth switching to a maintained version, e.g. [mtrojnar's](https://github.com/mtrojnar/osslsigncode) one.`osslsigncode` on SoureForge seems to be dead for a while now. It's worth switching to a maintained version, e.g. [mtrojnar's](https://github.com/mtrojnar/osslsigncode) one.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/32477Clean up obfs4/go build on Android2023-01-05T14:16:33ZboklmClean up obfs4/go build on AndroidWith legacy/trac#28803 (commit eee5d30a9ab1d727caac262cb62f72aaab75e0a0), we added support for Android for the obfs4 build.
In some of the go dependencies, we setup `var/compiler` (for example `gobsaes`), but not in some others (for exa...With legacy/trac#28803 (commit eee5d30a9ab1d727caac262cb62f72aaab75e0a0), we added support for Android for the obfs4 build.
In some of the go dependencies, we setup `var/compiler` (for example `gobsaes`), but not in some others (for example `ed25519`).
We should:
- understand why it is needed for some projects and not others
- check if it is really needed for all the projects where we add it
- understand why it is needed for Android, but not for the other platformshttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/31517Simplify macOS related bits in Firefox project2023-01-05T12:45:16ZGeorg KoppenSimplify macOS related bits in Firefox projectlegacy/trac#30323 included already simplifications made in https://bugzilla.mozilla.org/show_bug.cgi?id=1513798 but not all of them due to breakage. We should follow-up on that and bring our compile instructions closer to what Mozilla is...legacy/trac#30323 included already simplifications made in https://bugzilla.mozilla.org/show_bug.cgi?id=1513798 but not all of them due to breakage. We should follow-up on that and bring our compile instructions closer to what Mozilla is deploying.boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/29615Adjust creation of buildID script2023-02-01T12:10:28ZGeorg KoppenAdjust creation of buildID scriptWe should adjust the creation of our build ID script to make sure we have a larger space using the months available (currently Tor Browser 17 is the last major version that produces valid buildIDs). And we should think about a good way t...We should adjust the creation of our build ID script to make sure we have a larger space using the months available (currently Tor Browser 17 is the last major version that produces valid buildIDs). And we should think about a good way to implement something where we don't need to worry about the buildID creation in the future anymore.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/29041Compile clang closer to how Mozilla does it2023-01-05T14:13:28ZGeorg KoppenCompile clang closer to how Mozilla does itWe compile clang differently to how Mozilla does it: Mozilla has an elaborate setup with three stages (if needed) (see: https://hg.mozilla.org/releases/mozilla-esr60/file/248ca5c585f8/build/build-clang/build-clang.py) while we essentiall...We compile clang differently to how Mozilla does it: Mozilla has an elaborate setup with three stages (if needed) (see: https://hg.mozilla.org/releases/mozilla-esr60/file/248ca5c585f8/build/build-clang/build-clang.py) while we essentially "just" do what is outlined on https://clang.llvm.org/get_started.html.
We should change that and get our toolchain closer to what Mozilla provides.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/28595Remove the need to update var/gradle_dependencies_version2022-08-03T12:38:51ZboklmRemove the need to update var/gradle_dependencies_versionCurrently we need to update `var/gradle_dependencies_version` each time `gradle-dependencies-list.txt` is updated.
If you change `gradle-dependencies-list.txt` without updating `var/gradle_dependencies_version`, then you can easily end ...Currently we need to update `var/gradle_dependencies_version` each time `gradle-dependencies-list.txt` is updated.
If you change `gradle-dependencies-list.txt` without updating `var/gradle_dependencies_version`, then you can easily end up with the wrong list of gradle dependencies being used in a build.
To avoid this we should see if we can make the gradle dependencies be fetched again automatically when `gradle-dependencies-list.txt` has been updated, without the need to update `var/gradle_dependencies_version`.boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/25863Check where the -mwindows flag is needed2023-01-05T12:42:49ZboklmCheck where the -mwindows flag is neededCurrently we are setting the `-mwindows` flag by default in `CFLAGS` and `LDFLAGS` defined in `rbm.conf`, which are currently used (through `var/configure_opt`) in tor, gmp, libevent and go.
We should check where this flag is really nee...Currently we are setting the `-mwindows` flag by default in `CFLAGS` and `LDFLAGS` defined in `rbm.conf`, which are currently used (through `var/configure_opt`) in tor, gmp, libevent and go.
We should check where this flag is really needed, and only set it there.Sponsor 131 - Phase 5 - Ongoing Maintenance