tor-browser-build issueshttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues2023-04-18T13:18:09Zhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40831Fix update URL for base-browser nightly2023-04-18T13:18:09ZboklmFix update URL for base-browser nightlyThe update URLs for base-browser look like this: https://nightlies.tbb.torproject.org/nightly-updates/updates/basebrowser-nightly-linux-x86_64/
However, we currently set it to https://nightlies.tbb.torproject.org/nightly-updates/updates...The update URLs for base-browser look like this: https://nightlies.tbb.torproject.org/nightly-updates/updates/basebrowser-nightly-linux-x86_64/
However, we currently set it to https://nightlies.tbb.torproject.org/nightly-updates/updates/nightly-basebrowser-linux-x86_64/boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40830The fontconfig directory is missing in Base Browser2023-05-30T16:13:07ZPier Angelo VendrameThe fontconfig directory is missing in Base BrowserWe're missing the `fontconfig` directory in Base Browser.
It's a regression from that time that I thought that `projects/browser/Bundle-Data/$os` contained Tor Browser-only files.
However, I don't think we have to add the `fontconfig` ...We're missing the `fontconfig` directory in Base Browser.
It's a regression from that time that I thought that `projects/browser/Bundle-Data/$os` contained Tor Browser-only files.
However, I don't think we have to add the `fontconfig` directory in a `$ProjectName` subdirectory of `Browser`.Pier Angelo VendramePier Angelo Vendramehttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40829Review and standardize naming scheme for browser installer/package artifacts2023-12-07T13:44:11ZrichardReview and standardize naming scheme for browser installer/package artifactsWe should try to come up with some standard naming scheme (eg a like a target triple) for our various installers and packages, especially as the possibility for building new architectures is coming down the pipe.
We should drop the ALL ...We should try to come up with some standard naming scheme (eg a like a target triple) for our various installers and packages, especially as the possibility for building new architectures is coming down the pipe.
We should drop the ALL locale since there is no need to differentiate by locale.boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40828Use http://archive.debian.org/debian-archive/ for jessie2023-05-05T12:02:14ZboklmUse http://archive.debian.org/debian-archive/ for jessieIt seems jessie is now removed from the main debian mirrors:
```
W: Failed to fetch http://security.debian.org/debian-security/dists/jessie/updates/main/binary-amd64/Packages 404 Not Found [IP: 2a04:4e42:600::644 80]
W: Failed to fetc...It seems jessie is now removed from the main debian mirrors:
```
W: Failed to fetch http://security.debian.org/debian-security/dists/jessie/updates/main/binary-amd64/Packages 404 Not Found [IP: 2a04:4e42:600::644 80]
W: Failed to fetch http://deb.debian.org/debian/dists/jessie/main/binary-amd64/Packages 404 Not Found [IP: 2a04:4e42:3a::644 80]
W: Failed to fetch http://deb.debian.org/debian/dists/jessie-updates/main/binary-amd64/Packages 404 Not Found [IP: 2a04:4e42:3a::644 80]
```
Until #40102 is done, we can use http://archive.debian.org/debian-archive/.boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40827MAR generation uses (mostly) hard-coded MAR update channel2023-04-17T20:25:19ZrichardMAR generation uses (mostly) hard-coded MAR update channelEvery MAR file has an associated `MAR_CHANNEL_ID` which consists of a triple in the form `$BROWSER_NAME-$PUBLISHER-$CHANNEL`. For tor-browser this is `torbrowser-torproject-(alpha|release|nightly)`. At update-time the browser checks this...Every MAR file has an associated `MAR_CHANNEL_ID` which consists of a triple in the form `$BROWSER_NAME-$PUBLISHER-$CHANNEL`. For tor-browser this is `torbrowser-torproject-(alpha|release|nightly)`. At update-time the browser checks this string against its own stored in update-settings.ini
Adding base-browser to the mix causes a problem as the firefox build process will correctly update the `update-settings.ini` file to `basebrowser-torproject-(alpha|release|nightly)` but the MAR generation scripts use the hardcoded `torbrowser-torproejct-` prefix.richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40826Correctly set appname_marfile for basebrowser in tools/signing/nightly/update...2023-04-17T20:25:06ZboklmCorrectly set appname_marfile for basebrowser in tools/signing/nightly/update-responses-base-config.ymlboklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40825Try SOURCE_DATE_EPOCH in nsis2023-03-29T17:15:53ZPier Angelo VendrameTry SOURCE_DATE_EPOCH in nsisI've tried to upstream the `--no-insert-timestamp` patch we use to make NSIS reproducible.
However, I've been answered to use `SOURCE_DATE_EPOCH`, instead (added in NSIS 3.06.1).
We could try that, and drop our patch if it works.I've tried to upstream the `--no-insert-timestamp` patch we use to make NSIS reproducible.
However, I've been answered to use `SOURCE_DATE_EPOCH`, instead (added in NSIS 3.06.1).
We could try that, and drop our patch if it works.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40824dmg2mar script using hardcoded project names for paths2023-04-18T17:27:19Zricharddmg2mar script using hardcoded project names for pathsWe can probably overide these env variables in the `projects/release/dmg2mar` script when invoking `tools/dmg2mar`
```
# If the application is not TorBrowser (for instance, TorMessenger)
# set the application name in the TOR_APPNAME_BUN...We can probably overide these env variables in the `projects/release/dmg2mar` script when invoking `tools/dmg2mar`
```
# If the application is not TorBrowser (for instance, TorMessenger)
# set the application name in the TOR_APPNAME_BUNDLE_OSX,
# TOR_APPNAME_DMGFILE and TOR_APPNAME_MARFILE environment variables
my $appname = $ENV{TOR_APPNAME_BUNDLE_OSX} // 'Tor Browser';
my $appname_dmg = $ENV{TOR_APPNAME_DMGFILE} // 'TorBrowser';
my $appname_mar = $ENV{TOR_APPNAME_MARFILE} // 'tor-browser';
```https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40823Update appname_* variables in projects/release/update_responses_config.yml2023-04-17T20:21:24ZboklmUpdate appname_* variables in projects/release/update_responses_config.ymlWe need to update the various appname_* variables in projects/release/update_responses_config.yml, to make that work with other browser names.We need to update the various appname_* variables in projects/release/update_responses_config.yml, to make that work with other browser names.richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40822The Tor Browser installer doesn't run with mandatory ASLR on (0xc000007b)2023-05-02T14:16:52ZPier Angelo VendrameThe Tor Browser installer doesn't run with mandatory ASLR on (0xc000007b)We have received some feedback that Tor Browser can't be installed on Windows because of some error with code `0xc000007b`.
Now we know that it is caused by mandatory ASLR turned on on some machines:
1. Open Settings
2. Search for Expl...We have received some feedback that Tor Browser can't be installed on Windows because of some error with code `0xc000007b`.
Now we know that it is caused by mandatory ASLR turned on on some machines:
1. Open Settings
2. Search for Exploit protection in the left search bar
3. _Force randomization for Images (Mandatory ASLR)_ (it should be the third setting):
- Set it to _On by default_ to test the problem
- Set it to _Off by default_ (which is usually the default) if it's on and you want to install Tor Browser.
Please notice that the browser binaries work with this setting, and only the first installation fails.
However, changing the settings requires admin privileges and a reboot.
So, while it's a workaround, it might not always be feasible.
As for a real solution, I've already tried to pass `-Wl,--dynamicbase` when building NSIS.
My conjecture is that `makensis` doesn't actually compile an exe, but appends stuff to the exe we build earlier.
It wasn't trivial, and it wasn't effective either.
NSIS uses scons, I've tried with the `APPEND_LINKFLAGS` variable, but it didn't work (it stops finding zlib, when using this flag).
So, I've replaced `[% c("arch") %]-w64-mingw32-g++` with a script that invokes the real mingw g++ with the flags we need (I've copied the way from the Go `build` script).
FWIW, using MS tooling (from the VS cmd), I could see that our installers were already detected as with dynamic base on.
I think that GCC/binutils now have `--dynamicbase` enabled by default (or it takes the flag that we pass - I thought they were for building the tools only).
Passing `-Wl,--disable-dynamicbase` in the way I've described above didn't work, either.
But `dumpbin /headers` still lists "Dynamic base" under "Optional header values".
I've checked also `Firefox Installer.exe` (the stub that downloads the actual installer): it doesn't contain "Dynamic base" (but it isn't an NSIS installer, either).Pier Angelo VendramePier Angelo Vendramehttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40821The update details URL is wrong in alphas2023-10-03T15:35:41ZPier Angelo VendrameThe update details URL is wrong in alphasI've tried to check the update details in the just updated 12.5a4, but it sent me to a wrong URL: https://blog.torproject.org/new-release-tor-browser-125a4, instead of https://blog.torproject.org/new-alpha-release-tor-browser-125a4/ (`al...I've tried to check the update details in the just updated 12.5a4, but it sent me to a wrong URL: https://blog.torproject.org/new-release-tor-browser-125a4, instead of https://blog.torproject.org/new-alpha-release-tor-browser-125a4/ (`alpha` is missing).
This URL is generated with the info form `projects/release/update_responses_config.yml`.richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40820Prepare Tor Browser Stable 12.0.62023-06-15T07:29:01ZrichardPrepare Tor Browser Stable 12.0.6<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building tor-browser tags, labels, etc
- example : `91.6.0`
- `$(ESR_TAG)` : the Mozilla defined hg (Mercurial) tag associated with `$(ESR_VERSION)`
- exmaple : `FIREFOX_91_7_0esr_BUILD2`
- `$(ESR_TAG_PREV)` : the Mozilla defined hg (Mercurial) tag associated with the previous ESR version when rebasing (ie, the ESR version we are rebasing from)
- `$(TOR_BROWSER_MAJOR)` : the Tor Browser major version
- example : `11`
- `$(TOR_BROWSER_MINOR)` : the Tor Browser minor version
- example : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(TOR_BROWSER_VERSION)` : the Tor Browser version in the format
- example: `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(TOR_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- example : `build1`
- `$(TOR_BROWSER_BUILD_N)` : the tor-browser build revision for a given Tor Browser release; used in tagging git commits
- example : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(TOR_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For example :
- if we have multiple Tor Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(TOR_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(TOR_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `tor-browser`, the `$(TOR_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(TOR_BROWSER_VERSION)` : the published Tor Browser version
- example : `11.5a6`, `11.0.7`
</details>
**NOTE** It is assumed that the `tor-browser` rebase and security backport tasks have been completed
<details>
<summary>Build Configs</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Tor Browser Stable lives in the various `maint-$(TOR_BROWSER_MAJOR).$(TOR_BROWSER_MINOR)` (and possibly more specific) branches
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(TOR_BROWSER_BUILD_N)`
- [x] ***(Desktop Only)***`var/torbrowser_incremental_from` : update to previous Desktop version
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make incrementals-*` step will fail
- [x] Update Desktop-specific build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-release` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [x] `steps/base-browser-fluent/git_hash` : update with `HEAD` commit of project's `basebrowser-newidentityftl` branch
- [x] `steps/tor-browser/git_hash` : update with `HEAD` commit of project's `tor-browser` branch
- [x] `steps/fenix/git_hash` : update with `HEAD` commit of project's `fenix-torbrowserstringsxml` branch
- [x] Update Android-specific build configs
- [x] ***(Optional)*** Update `projects/geckoview/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/geckoview_version` : update to latest `$(ESR_VERSION)` if rebased
- [ ] ***(Optional)*** Update `projects/tor-android-service/config`
- [ ] `git_hash` : update with `HEAD` commit of project's `main` branch
- [ ] ***(Optional)*** Update `projects/application-services/config`:
**NOTE** we don't currently have any of our own patches for this project
- [ ] `git_hash` : update to appropriate git commit associated with `$(ESR_VERSION)`
- [ ] ***(Optional)*** Update `projects/android-components/config`:
- [ ] `android_components_build` : update to match android-components tag
- [ ] ***(Optional)*** Update `projects/fenix/config`
- [ ] `fenix_build` : update to match fenix tag
- [ ] ***(Optional)*** `var/fenix_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update allowed_addons.json by running (from `tor-browser-build` root):
- `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
- [ ] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for OpenSSL updates here : https://www.openssl.org/source/
- [ ] ***(Optional)*** If new 1.X.Y version available, update `projects/openssl/config`
- [ ] `version` : update to next 1.X.Y version
- [ ] `input_files/sha256sum` : update to sha256 sum of source tarball
- [x] Check for zlib updates here: https://github.com/madler/zlib/releases
- [ ] **(Optional)** If new tag available, update `projects/zlib/config`
- [ ] `version` : update to next release tag
- [x] Check for tor updates here : https://gitlab.torproject.org/tpo/core/tor/-/tags
- [ ] ***(Optional)*** Update `projects/tor/config`
- [ ] `version` : update to latest non `-alpha` tag (ping dgoulet or ahf if unsure)
- [x] Check for go updates here : https://golang.org/dl
- **NOTE** : Tor Browser Stable uses the latest of the *previous* Stable major series go version (apart from the transition phase from Tor Browser Alpha to Stable, in which case Tor Browser Stable may use the latest major series go version)
- [x] ***(Optional)*** Update `projects/go/config`
- [ ] `version` : update go version
- [ ] `input_files/sha256sum` for `go` : update sha256sum of archive (sha256 sums are displayed on the go download page)
- [x] Update the manual : https://gitlab.torproject.org/tpo/web/manual/-/jobs/
- [x] Download the `artifacts.zip` file from latest build stage row (download icon button on the right)
- [x] Rename it to `manual_$PIPELINEID.zip`
- [x] Upload it to people.tpo
- [x] Update `projects/manual/config`
- [x] Change the version to `$PIPELINEID`
- [x] Update the hash in the input_files section
- [x] Update the URL if you have uploaded to a different people.tpo home
- [x] Update `ChangeLog.txt`
- [x] Ensure ChangeLog.txt is sync'd between alpha and stable branches
- [x] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [x] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [x] Copy the output of the script to the beginning of `ChangeLog.txt` and adjust its output
- If you used the issue number, you will need to write the Tor Browser version manually
- [x] Include any version updates for:
- [x] translations
- [ ] OpenSSL
- [ ] NoScript
- [x] Go
- [ ] zlib
- [x] Include any ESR rebase for Firefox and GeckoView
- [x] Open MR with above changes
- [x] Begin build on `$(BUILD_SERVER)` (and fix any issues which come up and update MR)
- [x] Merge
- [x] Sign/Tag commit: `make torbrowser-signtag-release`
- [x] Push tag to `origin`
</details>
<details>
<summary>Communications</summary>
### notify stakeholders
<details>
<summary>email template</summary>
Hello All,
Unsigned Tor Browser $(TOR_BROWSER_VERSION) release candidate builds are now available for testing:
- https://tb-build-05.torproject.org/~$(BUILDER)/builds/release/unsigned/$(TOR_BROWSER_VERSION)/
The full changelog can be found here:
- https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/raw/maint-12.0/projects/browser/Bundle-Data/Docs/ChangeLog.txt
</details>
- [x] Email tor-qa mailing list: tor-qa@lists.torproject.org
- Additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
- [x] Email downstream consumers:
- Recipients:
- Tails dev mailing list: tails-dev@boum.org
- Guardian Project: nathan@guardianproject.info
- torbrowser-launcher: micah@micahflee.com
- FreeBSD port: freebsd@sysctl.cz <!-- Gitlab user maxfx -->
- OpenBSD port: caspar@schutijser.com <!-- Gitlab user cschutijser -->
- [ ] Note any changes which may affect packaging/downstream integration
- [ ] Email upstream stakeholders:
- ***(Optional, after ESR migration)*** Cloudflare: ask-research@cloudflare.com
- **NOTE** : We need to provide them with updated user agent string so they can update their internal machinery to prevent Tor Browser users from getting so many CAPTCHAs
</details>
<details>
<summary>Signing</summary>
### signing + publishing
- [x] Ensure builders have matching builds
- [ ] On `$(STAGING_SERVER)`, ensure updated:
- [ ] `tor-browser-build/tools/signing/set-config`
- `NSS_DB_DIR` : location of the `nssdb7` directory
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a tor notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/tor-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : tor browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] apk signing : copy signed `*multi.apk` files to the unsigned build outputs directory
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.sh`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [x] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Enable update responses : `sudo -u tb-release ./deploy_update_responses-release.sh`
- [ ] Remove old release data from following places:
- **NOTE** : Skip this step if the current release is Android or Desktop *only*
- [ ] `/srv/cdn-master.torproject.org/htdocs/aus1/torbrowser`
- [ ] `/srv/dist-master.torproject.org/htdocs/torbrowser`
- [x] Static update components (again) : `static-update-component cdn.torproject.org && static-update-component
- [x] Publish APKs to Google Play:
- Log into https://play.google.com/apps/publish
- Select `Tor Browser` app
- Navigate to `Release > Production` and click `Create new release` button:
- Upload the `*.multi.apk` APKs
- Update Release Name to Tor Browser version number
- Update Release Notes
- Next to 'Release notes', click `Copy from a previous release`
- Edit blog post url to point to most recent blog post
- Save, review, and configure rollout percentage
- [x] 25% rollout when publishing a scheduled update
- [ ] 100% rollout when publishing a security-driven release
- [x] Update rollout percentage to 100% after confirmed no major issues
</details>
<details>
<summary>Publishing</summary>
### website: https://gitlab.torproject.org/tpo/web/tpo.git
- [x] `databags/versions.ini` : Update the downloads versions
- `torbrowser-stable/version` : sort of a catch-all for latest stable version
- `torbrowser-alpha/version` : sort of a catch-all for latest stable version
- `torbrowser-*-stable/version` : platform-specific stable versions
- `torbrowser-*-alpha/version` : platform-specific alpha versions
- `tor-stable`,`tor-alpha` : set by tor devs, do not touch
- [x] Push to origin as new branch, open 'Draft :' MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and builds are published
### blog: https://gitlab.torproject.org/tpo/web/blog.git
- [x] Duplicate previous Stable or Alpha release blog post as appropriate to new directory under `content/blog/new-release-tor-browser-$(TOR_BROWSER_VERSION)` and update with info on release :
- [x] Update Tor Browser version numbers
- [x] Note any ESR rebase
- [x] Link to any Firefox security updates from ESR upgrade
- [x] Link to any Android-specific security backports
- [x] Note any updates to :
- tor
- OpenSSL
- NoScript
- [x] Convert ChangeLog.txt to markdown format used here by :
- `tor-browser-build/tools/changelog-format-blog-post`
- [x] Push to origin as new branch, open `Draft:` MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and website has been updated
### tor-announce mailing list
- [x] Send an email to tor-announce@lists.torproject.org, using the same content as the blog post and subject "Tor Browser $version is released".
</details>richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40819Sign macOS tor executables in tor expert bundle2023-08-22T20:10:21ZsebSign macOS tor executables in tor expert bundleI'm trying to use the tor executable and pluggable transports from the tor expert bundle while porting briar-desktop to macOS. I've seen https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40397 and thanks for creat...I'm trying to use the tor executable and pluggable transports from the tor expert bundle while porting briar-desktop to macOS. I've seen https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40397 and thanks for creating the tor expert bundle in the first place!
It looks like the `tor` executable shipped with the expert bundle is not signed. As a result, I cannot run it as a subprocess from within the JVM. I've started debugging this and it looks like it doesn't have anything to do with the way we use `ProcessBuilder` to launch the executable on the JVM (everything works fine when I use the `tor.real` executable shipped with the TorBrowser DMG package). Taking the JVM side of things out of the picture, when I try to run `./tor` from the expert bundle on the shell, I do get this:
```
zsh: killed ./tor
```
It might also show a popup notifying me about the fact that the developer of the executable cannot be verified with the options in the dialog to either move it to trash or cancel the operation.
The executable shipped in the TorBrowser DMG packages works fine however. I wasn't sure it's actually the executable itself that is signed or if the OS keeps track of the DMG it has been extracted from (which is signed itself). So I extracted the file on a Linux machine and transferred it to a macOS machine that had never seen that file or TorBrowser before. I was still able to run `tor.real` successfully there.
This makes me wonder: would it be desirable from your point of view and technically possible to sign the executables shipped with the expert bundle the same way the ones from the TorBrowser distribution are?boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40818Enable wasm target for rust compiler2023-05-11T15:39:05ZCecylia BocovichEnable wasm target for rust compilerAfter the discussion in https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/116, I did a bit more investigation and it looks like we do need to enable the `wasm32-unknown-unknown` target in the rust project in order to compil...After the discussion in https://gitlab.torproject.org/tpo/anti-censorship/team/-/issues/116, I did a bit more investigation and it looks like we do need to enable the `wasm32-unknown-unknown` target in the rust project in order to compile the lox client to wasm in our reproducible build system.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40817Add basebrowser-incrementals-nightly makefile target2023-03-22T11:48:33ZboklmAdd basebrowser-incrementals-nightly makefile targetIn order to create incrementals for basebrowser nightly builds (tor-browser-bundle-testsuite#40073), we need to add a basebrowser-incrementals-nightly makefile target.In order to create incrementals for basebrowser nightly builds (tor-browser-bundle-testsuite#40073), we need to add a basebrowser-incrementals-nightly makefile target.boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40816Race condition in macos-signer-gatekeeper-signing script2023-08-26T03:04:20ZrichardRace condition in macos-signer-gatekeeper-signing scriptRan into this this morning, fortunately(?) it doesnt alwasy happen so a second run was successful.
Logs:
```
2023-03-18T10:26:28+00:00 - Starting step: macos-signer-gatekeeper-signing
building file list ... done
created directory 12.0....Ran into this this morning, fortunately(?) it doesnt alwasy happen so a second run was successful.
Logs:
```
2023-03-18T10:26:28+00:00 - Starting step: macos-signer-gatekeeper-signing
building file list ... done
created directory 12.0.4
TorBrowser-12.0.4-macos_ALL.dmg
sent 170,979,310 bytes received 75 bytes 8,340,457.80 bytes/sec
total size is 170,958,294 speedup is 1.00
Checksumming whole disk (unknown partition : 0)…
whole disk (unknown partition : 0): verified CRC32 $601C6983
verified CRC32 $CFD0707F
/dev/disk2 /Volumes/Tor Browser
Signing Tor Browser_ALL.app
Tor Browser.app/: signed app bundle with Mach-O universal (x86_64 arm64) [org.torproject.torbrowser]
codesign exit code: 0
Checking ALL...
Tor Browser.app/: rejected
source=Unnotarized Developer ID
origin=Developer ID Application: The Tor Project, Inc (MADPSAYN6T)
Tor Browser.app/: accepted
source=Developer ID
origin=Developer ID Application: The Tor Project, Inc (MADPSAYN6T)
Zipping up tb-12.0.4_ALL.zip
hdiutil: couldn't unmount "disk2" - Resource busy
```
To rectify, I had to log-in to mac-signer manually invoke the `hdiutil deatch ...` command and re-run (which succeeded on a second try).
Looking at the script I'm not quite sure what could have caused this. I've never seen this before so perhaps we can just ignore until we have macOS signing on Linux working vOv.https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40815Include platform details in some artifact filenames2024-01-10T08:30:59ZPier Angelo VendrameInclude platform details in some artifact filenamesSome artifacts are tied to a certain platform (e.g., Mingw, Rust, etc).
Sometimes knowing it at a glance could be useful (e.g., when reusing these artifacts outside tor-browser-build, e.g., to create a container for local incremental bu...Some artifacts are tied to a certain platform (e.g., Mingw, Rust, etc).
Sometimes knowing it at a glance could be useful (e.g., when reusing these artifacts outside tor-browser-build, e.g., to create a container for local incremental builds).
We could do that when updating the toolchains for the next ESR.
List of artifacts to fix:
- [x] ~~`mingw-w64-clang`~~ -> switched to single package for both 32-bit and 64-bit
- [x] Rust
- [ ] Binutilshttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40814Prepare Tor Browser Stable 12.0.52023-06-15T07:29:01ZrichardPrepare Tor Browser Stable 12.0.5<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building tor-browser tags, labels, etc
- example : `91.6.0`
- `$(ESR_TAG)` : the Mozilla defined hg (Mercurial) tag associated with `$(ESR_VERSION)`
- exmaple : `FIREFOX_91_7_0esr_BUILD2`
- `$(ESR_TAG_PREV)` : the Mozilla defined hg (Mercurial) tag associated with the previous ESR version when rebasing (ie, the ESR version we are rebasing from)
- `$(TOR_BROWSER_MAJOR)` : the Tor Browser major version
- example : `11`
- `$(TOR_BROWSER_MINOR)` : the Tor Browser minor version
- example : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(TOR_BROWSER_VERSION)` : the Tor Browser version in the format
- example: `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(TOR_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- example : `build1`
- `$(TOR_BROWSER_BUILD_N)` : the tor-browser build revision for a given Tor Browser release; used in tagging git commits
- example : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(TOR_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For example :
- if we have multiple Tor Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(TOR_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(TOR_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `tor-browser`, the `$(TOR_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(TOR_BROWSER_VERSION)` : the published Tor Browser version
- example : `11.5a6`, `11.0.7`
</details>
**NOTE** It is assumed that the `tor-browser` rebase and security backport tasks have been completed
<details>
<summary>Build Configs</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Tor Browser Stable lives in the various `maint-$(TOR_BROWSER_MAJOR).$(TOR_BROWSER_MINOR)` (and possibly more specific) branches
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(TOR_BROWSER_BUILD_N)`
- [x] ***(Desktop Only)***`var/torbrowser_incremental_from` : update to previous Desktop version
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make incrementals-*` step will fail
- [ ] Update Desktop-specific build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [x] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-release` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [x] `steps/base-browser-fluent/git_hash` : update with `HEAD` commit of project's `basebrowser-newidentityftl` branch
- [x] `steps/tor-browser/git_hash` : update with `HEAD` commit of project's `tor-browser` branch
- [x] `steps/fenix/git_hash` : update with `HEAD` commit of project's `fenix-torbrowserstringsxml` branch
- [ ] Update Android-specific build configs
- [x] ***(Optional)*** Update `projects/geckoview/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [x] ***(Optional)*** `var/geckoview_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] ***(Optional)*** Update `projects/tor-android-service/config`
- [x] `git_hash` : update with `HEAD` commit of project's `main` branch
- [ ] ***(Optional)*** Update `projects/application-services/config`:
**NOTE** we don't currently have any of our own patches for this project
- [ ] `git_hash` : update to appropriate git commit associated with `$(ESR_VERSION)`
- [ ] ***(Optional)*** Update `projects/android-components/config`:
- [ ] `android_components_build` : update to match `android-components` tag
- [ ] ***(Optional)*** Update `projects/fenix/config`
- [ ] `fenix_build` : update to match `fenix` tag
- [ ] ***(Optional)*** `var/fenix_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update allowed_addons.json by running (from `tor-browser-build` root):
- `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
- [ ] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [x] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [x] `URL`
- [x] `sha256sum`
- [x] Check for OpenSSL updates here : https://www.openssl.org/source/
- [ ] ***(Optional)*** If new 1.X.Y version available, update `projects/openssl/config`
- [ ] `version` : update to next 1.X.Y version
- [ ] `input_files/sha256sum` : update to sha256 sum of source tarball
- [x] Check for zlib updates here: https://github.com/madler/zlib/releases
- [ ] **(Optional)** If new tag available, update `projects/zlib/config`
- [ ] `version` : update to next release tag
- [x] Check for tor updates here : https://gitlab.torproject.org/tpo/core/tor/-/tags
- [ ] ***(Optional)*** Update `projects/tor/config`
- [ ] `version` : update to latest non `-alpha` tag (ping dgoulet or ahf if unsure)
- [x] Check for go updates here : https://golang.org/dl
- **NOTE** : Tor Browser Stable uses the latest of the *previous* Stable major series go version (apart from the transition phase from Tor Browser Alpha to Stable, in which case Tor Browser Stable may use the latest major series go version)
- [x] ***(Optional)*** Update `projects/go/config`
- [x] `version` : update go version
- [x] `input_files/sha256sum` for `go` : update sha256sum of archive (sha256 sums are displayed on the go download page)
- [x] Update the manual : https://gitlab.torproject.org/tpo/web/manual/-/jobs/
- [x] Download the `artifacts.zip` file from latest build stage row (download icon button on the right)
- [x] Rename it to `manual_$PIPELINEID.zip`
- [x] Upload it to people.tpo
- [x] Update `projects/manual/config`
- [x] Change the version to `$PIPELINEID`
- [x] Update the hash in the input_files section
- [x] Update the URL if you have uploaded to a different people.tpo home
- [x] Update `ChangeLog.txt`
- [x] Ensure ChangeLog.txt is sync'd between alpha and stable branches
- [ ] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [x] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [x] Copy the output of the script to the beginning of `ChangeLog.txt` and adjust its output
- If you used the issue number, you will need to write the Tor Browser version manually
- [ ] Include any version updates for:
- [x] translations
- [ ] OpenSSL
- [x] NoScript
- [x] Go
- [ ] zlib
- [x] Include any ESR rebase for Firefox and GeckoView
- [x] Open MR with above changes
- [x] Begin build on `$(BUILD_SERVER)` (and fix any issues which come up and update MR)
- [x] Merge
- [x] Sign/Tag commit: `make signtag-release`
- [x] Push tag to `origin`
</details>
<details>
<summary>Communications</summary>
### notify stakeholders
<details>
<summary>email template</summary>
Hello All,
Unsigned Tor Browser $(TOR_BROWSER_VERSION) release candidate builds are now available for testing:
- https://tb-build-05.torproject.org/~$(BUILDER)/builds/release/unsigned/$(TOR_BROWSER_VERSION)/
The full changelog can be found here:
- https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/raw/maint-12.0/projects/browser/Bundle-Data/Docs/ChangeLog.txt
</details>
- [ ] Email tor-qa mailing list: tor-qa@lists.torproject.org
- Additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
- [ ] Email downstream consumers:
- Recipients:
- Tails dev mailing list: tails-dev@boum.org
- Guardian Project: nathan@guardianproject.info
- torbrowser-launcher: micah@micahflee.com
- FreeBSD port: freebsd@sysctl.cz <!-- Gitlab user maxfx -->
- OpenBSD port: caspar@schutijser.com <!-- Gitlab user cschutijser -->
- [ ] Note any changes which may affect packaging/downstream integration
- [ ] Email upstream stakeholders:
- ***(Optional, after ESR migration)*** Cloudflare: ask-research@cloudflare.com
- **NOTE** : We need to provide them with updated user agent string so they can update their internal machinery to prevent Tor Browser users from getting so many CAPTCHAs
</details>
<details>
<summary>Signing</summary>
### signing + publishing
- [x] Ensure builders have matching builds
- [x] On `$(STAGING_SERVER)`, ensure updated:
- [ ] `tor-browser-build/tools/signing/set-config`
- `NSS_DB_DIR` : location of the `nssdb7` directory
- [ ] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [ ] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a tor notariser Apple Developer account
- [ ] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/tor-browser-update-responses.git`
- [ ] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : tor browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [ ] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [ ] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [ ] apk signing : copy signed `*multi.apk` files to the unsigned build outputs directory
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.sh`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [ ] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Enable update responses : `sudo -u tb-release ./deploy_update_responses-release.sh`
- [ ] Remove old release data from following places:
- **NOTE** : Skip this step if the current release is Android or Desktop *only*
- [ ] `/srv/cdn-master.torproject.org/htdocs/aus1/torbrowser`
- [ ] `/srv/dist-master.torproject.org/htdocs/torbrowser`
- [ ] Static update components (again) : `static-update-component cdn.torproject.org && static-update-component
- [x] Publish APKs to Google Play:
- Log into https://play.google.com/apps/publish
- Select `Tor Browser` app
- Navigate to `Release > Production` and click `Create new release` button:
- Upload the `*.multi.apk` APKs
- Update Release Name to Tor Browser version number
- Update Release Notes
- Next to 'Release notes', click `Copy from a previous release`
- Edit blog post url to point to most recent blog post
- Save, review, and configure rollout percentage
- [x] 25% rollout when publishing a scheduled update
- [ ] 100% rollout when publishing a security-driven release
- [ ] Update rollout percentage to 100% after confirmed no major issues
</details>
<details>
<summary>Publishing</summary>
### website: https://gitlab.torproject.org/tpo/web/tpo.git
- [x] `databags/versions.ini` : Update the downloads versions
- `torbrowser-stable/version` : sort of a catch-all for latest stable version
- `torbrowser-alpha/version` : sort of a catch-all for latest stable version
- `torbrowser-*-stable/version` : platform-specific stable versions
- `torbrowser-*-alpha/version` : platform-specific alpha versions
- `tor-stable`,`tor-alpha` : set by tor devs, do not touch
- [x] Push to origin as new branch, open 'Draft :' MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and builds are published
### blog: https://gitlab.torproject.org/tpo/web/blog.git
- [x] Duplicate previous Stable or Alpha release blog post as appropriate to new directory under `content/blog/new-release-tor-browser-$(TOR_BROWSER_VERSION)` and update with info on release :
- [ ] Update Tor Browser version numbers
- [ ] Note any ESR rebase
- [ ] Link to any Firefox security updates from ESR upgrade
- [ ] Link to any Android-specific security backports
- [ ] Note any updates to :
- tor
- OpenSSL
- NoScript
- [ ] Convert ChangeLog.txt to markdown format used here by :
- `tor-browser-build/tools/changelog-format-blog-post`
- [x] Push to origin as new branch, open `Draft:` MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and website has been updated
### tor-announce mailing list
- [x] Send an email to tor-announce@lists.torproject.org, using the same content as the blog post and subject "Tor Browser $version is released".
</details>ma1ma1https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40813Enable var/updater_enabled for basebrowser nightly2023-03-20T18:45:13ZboklmEnable var/updater_enabled for basebrowser nightlyAfter tor-browser!577, the updater is enabled by default for base browser, so we should enable `var/updater_enabled` for basebrowser in `rbm.conf`.
Maybe we should also disable it for stable releases.After tor-browser!577, the updater is enabled by default for base browser, so we should enable `var/updater_enabled` for basebrowser in `rbm.conf`.
Maybe we should also disable it for stable releases.boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40812Make var/rezip in projects/firefox/config quiet2023-03-20T18:44:35ZboklmMake var/rezip in projects/firefox/config quiet`var/unzip` is using `unzip` to extract the zip file it is reziping. We should add a `-q` to unzip to avoid listing all the files in the build log.`var/unzip` is using `unzip` to extract the zip file it is reziping. We should add a `-q` to unzip to avoid listing all the files in the build log.boklmboklm