The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2023-09-29T21:46:31Zhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40967Prepare Tor Browser 12.5.6 stable release2023-09-29T21:46:31ZPier Angelo VendramePrepare Tor Browser 12.5.6 stable release<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** :...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building tor-browser tags, labels, etc
- **example** : `91.6.0`
- `$(TOR_BROWSER_MAJOR)` : the Tor Browser major version
- **example** : `11`
- `$(TOR_BROWSER_MINOR)` : the Tor Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(TOR_BROWSER_VERSION)` : the Tor Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(TOR_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(TOR_BROWSER_BUILD_N)` : the tor-browser build revision for a given Tor Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(TOR_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For example :
- if we have multiple Tor Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(TOR_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(TOR_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `tor-browser`, the `$(TOR_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(TOR_BROWSER_VERSION)` : the published Tor Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(TBB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Tor Browser version
- **example** : `tbb-12.0.7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` stable rebase and security backport tasks have been completed
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Tor Browser Stable lives in the various `maint-$(TOR_BROWSER_MAJOR).$(TOR_BROWSER_MINOR)` (and possibly more specific) branches
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [ ] `var/torbrowser_build` : update to `$(TOR_BROWSER_BUILD_N)`
- [x] ***(Desktop Only)***`var/torbrowser_incremental_from` : update to previous Desktop version
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make torbrowser-incrementals-*` step will fail
- [x] Update Desktop-specific build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-release` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [ ] `steps/base-browser-fluent/git_hash` : update with `HEAD` commit of project's `basebrowser-newidentityftl` branch
- [x] `steps/tor-browser/git_hash` : update with `HEAD` commit of project's `tor-browser` branch
- [x] `steps/fenix/git_hash` : update with `HEAD` commit of project's `fenix-torbrowserstringsxml` branch
- [x] Update Android-specific build configs
- [x] Update `projects/geckoview/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/geckoview_version` : update to latest `$(ESR_VERSION)` if rebased
- [ ] ***(Optional)*** Update `projects/tor-android-service/config`
- [ ] `git_hash` : update with `HEAD` commit of project's `main` branch
- [ ] ***(Optional)*** Update `projects/application-services/config`:
**NOTE** we don't currently have any of our own patches for this project
- [ ] `git_hash` : update to appropriate git commit associated with `$(ESR_VERSION)`
- [ ] ***(Optional)*** Update `projects/android-components/config`:
- [ ] `android_components_build` : update to match stable android-components tag
- [ ] ***(Optional)*** Update `projects/fenix/config`
- [ ] `fenix_build` : update to match fenix tag
- [x] Update allowed_addons.json by running (from `tor-browser-build` root):
- `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
- [ ] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for OpenSSL updates here : https://www.openssl.org/source/
- [ ] ***(Optional)*** If new 1.X.Y version available, update `projects/openssl/config`
- [ ] `version` : update to next 1.X.Y version
- [ ] `input_files/sha256sum` : update to sha256 sum of source tarball
- [x] Check for zlib updates here: https://github.com/madler/zlib/releases
- [ ] **(Optional)** If new tag available, update `projects/zlib/config`
- [ ] `version` : update to next release tag
- [x] Check for tor updates here : https://gitlab.torproject.org/tpo/core/tor/-/tags
- [ ] ***(Optional)*** Update `projects/tor/config`
- [ ] `version` : update to latest non `-alpha` tag (ping dgoulet or ahf if unsure)
- [x] Check for go updates here : https://golang.org/dl
- **NOTE** : Tor Browser Stable uses the latest of the *previous* Stable major series go version (apart from the transition phase from Tor Browser Alpha to Stable, in which case Tor Browser Stable may use the latest major series go version)
- [ ] ***(Optional)*** Update `projects/go/config`
- [ ] `version` : update go version
- [ ] `input_files/sha256sum` for `go` : update sha256sum of archive (sha256 sums are displayed on the go download page)
- [x] Check for manual updates by running (from `tor-browser-build` root): `./tools/fetch-manual.py`
- [ ] ***(Optional)*** If new version is available:
- [ ] Upload the downloaded `manual_$PIPELINEID.zip` file to people.tpo
- [ ] Update `projects/manual/config`:
- [ ] Change the `version` to `$PIPELINEID`
- [ ] Update `sha256sum` in the `input_files` section
- [ ] ***(Optional)*** Update the URL if you have uploaded to a different people.tpo home
- [x] Update `ChangeLog.txt`
- [x] Ensure ChangeLog.txt is sync'd between alpha and stable branches
- [ ] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [ ] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [ ] Copy the output of the script to the beginning of `ChangeLog.txt` and adjust its output
- **NOTE** : If you used the issue number, you will need to write the Tor Browser version manually
- [ ] ***(Optional)*** Under `All Platforms` include any version updates for:
- [ ] Translations
- [ ] OpenSSL
- [ ] NoScript
- [ ] zlib
- [ ] tor daemon
- [ ] ***(Optional)*** Under `Windows + macOS + Linux` include updates for:
- [ ] Firefox
- [ ] ***(Optional)*** Under `Android`, include updates for:
- [ ] Geckoview
- [ ] ***(Optional)*** Under `Build System/All Platforms` include updates for:
- [ ] Go
- [x] Open MR with above changes
- [x] Merge
- [x] Sign/Tag commit: `make torbrowser-signtag-release`
- [x] Push tag to `origin`
- [x] Begin build on `$(BUILD_SERVER)` (fix any issues in subsequent MRs)
- [ ] **TODO** Submit build-tag to Mullvad build infra
- [x] Ensure builders have matching builds
</details>
<details>
<summary>Communications</summary>
### notify stakeholders
<details>
<summary>email template</summary>
Subject:
Tor Browser $(TOR_BROWSER_VERION) (Android, Windows, macOS, Linux)
Body:
Hello All,
Unsigned Tor Browser $(TOR_BROWSER_VERSION) release candidate builds are now available for testing:
- https://tb-build-05.torproject.org/~$(BUILDER)/builds/release/unsigned/$(TOR_BROWSER_VERSION)/
The full changelog can be found here:
- https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/$(TBB_BUILD_TAG)/ChangeLog.txt
</details>
- [x] Email tor-qa mailing list: tor-qa@lists.torproject.org
- ***(Optional)*** Additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
- [x] Email packagers:
- Recipients:
- Tails dev mailing list: tails-dev@boum.org
- Guardian Project: nathan@guardianproject.info
- torbrowser-launcher: micah@micahflee.com
- FreeBSD port: freebsd@sysctl.cz <!-- Gitlab user maxfx -->
- OpenBSD port: caspar@schutijser.com <!-- Gitlab user cschutijser -->
- [ ] ***(Optional)*** Note any changes which may affect packaging/downstream integration
</details>
<details>
<summary>Signing</summary>
### signing
- **NOTE** : In practice, it's most efficient to have the blog post and website updates ready to merge, since signing doesn't take very long
- [x] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build` is on the right commit: `git tag -v tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N) && git checkout tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N)`
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a tor notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/tor-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : tor browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.torbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [ ] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Enable update responses : `sudo -u tb-release ./deploy_update_responses-release.sh`
- [ ] Remove old release data from following places:
- **NOTE** : Skip this step if we need to hold on to older versions for some reason (for example, this is an Andoid or Desktop-only release, or if we need to hold back installers in favor of build-to-build updates if there are signing issues, etc)
- [ ] `/srv/cdn-master.torproject.org/htdocs/aus1/torbrowser`
- [ ] `/srv/dist-master.torproject.org/htdocs/torbrowser`
- [ ] Static update components (again) : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Publish APKs to Google Play:
- Log into https://play.google.com/apps/publish
- Select `Tor Browser` app
- Navigate to `Release > Production` and click `Create new release` button:
- Upload the `*.multi.apk` APKs
- Update Release Name to Tor Browser version number
- Update Release Notes
- Next to 'Release notes', click `Copy from a previous release`
- Edit blog post url to point to most recent blog post
- Save, review, and configure rollout percentage
- [ ] 25% rollout when publishing a scheduled update
- [x] 100% rollout when publishing a security-driven release
- [ ] Update rollout percentage to 100% after confirmed no major issues
</details>
<details>
<summary>Signature verification</summary>
<details>
<summary>Check whether the .exe files got properly signed and timestamped</summary>
```
# Point OSSLSIGNCODE to your osslsigncode binary
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
OSSLSIGNCODE=/path/to/osslsigncode
../../../tools/authenticode_check.sh
popd
```
</details>
<details>
<summary>Check whether the MAR files got properly signed</summary>
```
# Point NSSDB to your nssdb containing the mar signing certificate
# Point SIGNMAR to your signmar binary
# Point LD_LIBRARY_PATH to your mar-tools directory
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
NSSDB=/path/to/nssdb
SIGNMAR=/path/to/mar-tools/signmar
LD_LIBRARY_PATH=/path/to/mar-tools/
../../../tools/marsigning_check.sh
popd
```
</details>
</details>
<details>
<summary>Publishing</summary>
### website: https://gitlab.torproject.org/tpo/web/tpo.git
- [x] `databags/versions.ini` : Update the downloads versions
- `torbrowser-stable/version` : sort of a catch-all for latest stable version
- `torbrowser-alpha/version` : sort of a catch-all for latest stable version
- `torbrowser-*-stable/version` : platform-specific stable versions
- `torbrowser-*-alpha/version` : platform-specific alpha versions
- `tor-stable`,`tor-alpha` : set by tor devs, do not touch
- [x] Push to origin as new branch, open 'Draft :' MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and builds are published
### blog: https://gitlab.torproject.org/tpo/web/blog.git
- [x] Duplicate previous Stable or Alpha release blog post as appropriate to new directory under `content/blog/new-release-tor-browser-$(TOR_BROWSER_VERSION)` and update with info on release :
- [x] Run `tools/signing/create-blog-post` which should create the new blog post from a template (edit set-config.blog to set you local blog directory)
- [ ] Update Tor Browser version numbers
- [ ] Note any ESR rebase
- [ ] Link to any Firefox security updates from ESR upgrade
- [ ] Link to any Android-specific security backports
- [ ] Note any updates to :
- tor
- OpenSSL
- NoScript
- [ ] Convert ChangeLog.txt to markdown format used here by :
- `tor-browser-build/tools/changelog-format-blog-post`
- [x] Push to origin as new branch, open `Draft:` MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and website has been updated
### tor-announce mailing list
<details>
<summary>email template</summary>
Subject:
New Release: Tor Browser $(TOR_BROWSER_VERSION) (Android, Windows, macOS, Linux)
Body:
Hi everyone,
Tor Browser $(TOR_BROWSER_VERSION) has now been published for all platforms. For details please see our blog post:
- $(BLOG_POST_URL)
</details>
- [x] Email tor-announce mailing list: tor-announce@lists.torproject.org
- **(Optional)** Additional information:
- [ ] Link to any known issues
</details>richardrichardhttps://gitlab.torproject.org/tpo/network-health/metrics/onionoo/-/issues/40039Onionoo seems to mix up data during two consecutive update runs2023-10-12T11:47:35ZGeorg KoppenOnionoo seems to mix up data during two consecutive update runs`bridgestrap` results are getting used once a day to overwrite the running status of bridges (see: https://gitlab.torproject.org/tpo/network-health/team/-/issues/318#note_2947956 for context). This is happening with the Onionoo updater r...`bridgestrap` results are getting used once a day to overwrite the running status of bridges (see: https://gitlab.torproject.org/tpo/network-health/team/-/issues/318#note_2947956 for context). This is happening with the Onionoo updater run at 1000UTC as the files are currently made public on CollecTor on around 0940UTC. This can be seen at the Onionoo log from today, 09/28/2023:
```
2023-09-28 10:07:28,664 INFO o.t.m.o.u.StatusUpdateRunner:51 NodeDetailsStatusUpdater
2 relay consensuses processed
2 bridge statuses processed
1 bridgestrap stats processed
```
The result of that can be seen in the following two screenshots (the first shows bridges with firewalled obfs4 port as running on relay-search, while the second one shows them as offline, even though they are running fine all the time):
![toralf4_10_49_09_28_2023_v2](/uploads/2996f6efd47f1980055016fb2fa77165/toralf4_10_49_09_28_2023_v2.png)![toralf4_11_16_09_28_2023_v2](/uploads/0b3fca6dcadbf14a979e02901f46513c/toralf4_11_16_09_28_2023_v2.png)
I retrieved the former 1049UTC and the latter 1116UTC, so the screenshots are as expected, given the `bridgestrap` results parsing at 1000UTC only.
But have a look at the data Onionoo claims is powering those screenshots: It's 0903UTC for the former and 1003UTC for the latter, which does not make sense given that at 0903UTC the `bridgestrap` test results are not even available yet. Somehow Onionoo data is getting mixed up within two runs it seems?Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/network-health/tor-weather/-/issues/79Make it easier do develop using Docker2023-10-02T08:05:16ZGeorg KoppenMake it easier do develop using DockerAdding an option for a Docker environment for dev purposes would make it easier for developers to work on Tor Weather.Adding an option for a Docker environment for dev purposes would make it easier for developers to work on Tor Weather.Sarthik Guptasarthikg@icloud.comSarthik Guptasarthikg@icloud.comhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42133Remove "Total Cookie Protection" popup2023-10-04T18:44:39ZclairehurstRemove "Total Cookie Protection" popup<!--
* Use this issue template for reporting a new bug.
-->
### Summary
**Summarize the bug encountered concisely.**
On a fresh install, turn on cookie banner protection and then open a site (I did youtube.com)
### Steps to reproduce...<!--
* Use this issue template for reporting a new bug.
-->
### Summary
**Summarize the bug encountered concisely.**
On a fresh install, turn on cookie banner protection and then open a site (I did youtube.com)
### Steps to reproduce:
**How one can reproduce the issue - this is very important.**
1. Get a fresh install of tor browser for android
2. Connect to tor and visit a site (I did youtube.com, I'm not sure if the site matters)
3. Turn on cookie banner protection
4. Go back to the site, (it should reload)
5. Notice popup near the bottom of the screen about "Total Cookie Protection"
Video of reproduction **Warning: flashing text and icons at the end" https://share.riseup.net/#-UqKADXExnqlAUqdPnpiqg
### What is the current bug behavior?
**What actually happens.**
There is a popup about "Total Cookie Protection"
### What is the expected behavior?
**What you want to see instead**
There should be not be Popup, we do not use this feature.
### Environment
**Which operating system are you using? For example: Debian GNU/Linux 10.1, Windows 10, Ubuntu Xenial, FreeBSD 12.2, etc.**
**Which installation method did you use? Distribution package (apt, pkg, homebrew), from source tarball, from Git, etc.**
macOS Ventura 13.5.2, Pixel 7 Pro API 34 emulator
### Relevant logs and/or screenshots
![image](/uploads/5a1d84296645114eacd380884fe8892d/image.png)
Tapping on "Learn more" results in this
**Warning!! flashing text and Icons**
![Screen_Recording_2023-09-27_at_11.52.38_AM](/uploads/2f417decb380f5a6e78e9d4ce1032fd1/Screen_Recording_2023-09-27_at_11.52.38_AM.mov){width=50%}clairehurstclairehursthttps://gitlab.torproject.org/tpo/core/arti/-/issues/1052Follow-up from "Draft: arti-client: sketch out an initial "launch an onion se...2023-09-27T17:28:43Zgabi-250Follow-up from "Draft: arti-client: sketch out an initial "launch an onion service" API."The following discussion from !1620 should be addressed:
- [ ] @gabi-250 started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1620#note_2947901):
> ```
> // TODO HSS: This is actually a Publi...The following discussion from !1620 should be addressed:
- [ ] @gabi-250 started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1620#note_2947901):
> ```
> // TODO HSS: This is actually a PublisherError, but that type isn't exposed,
> // and it contains a whole ecosystem of other crate-internal errors.
> // Either we should change Publisher::launch() to return a StartupError,
> // or we should figure out how much of PublisherError to expose.
> ```gabi-250gabi-250https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42130Add support for specifying the branch in `tb-dev rebase-on-default`2023-10-03T15:36:02ZhenryAdd support for specifying the branch in `tb-dev rebase-on-default``tb-dev rebase-on-default` is useful when you want to rebase on a new upstream branch. Currently, it only rebases the currently checked-out branch. It would be useful if it could also work on any specified branch.`tb-dev rebase-on-default` is useful when you want to rebase on a new upstream branch. Currently, it only rebases the currently checked-out branch. It would be useful if it could also work on any specified branch.henryhenryhttps://gitlab.torproject.org/tpo/team/-/issues/217Q3 Reports for DRL sponsored projects2023-10-30T13:45:49ZGabagaba@torproject.orgQ3 Reports for DRL sponsored projects61
- [x] Write the last report for the project
- [x] Gather indicators https://nc.torproject.net/f/462462
- [x] Send final docs to team for review
- [x] Send final docs to Bekeela and Isabela for review
96
- [x] Send mail [requesting i...61
- [x] Write the last report for the project
- [x] Gather indicators https://nc.torproject.net/f/462462
- [x] Send final docs to team for review
- [x] Send final docs to Bekeela and Isabela for review
96
- [x] Send mail [requesting info](http://kfahv6wfkbezjyg4r6mlhpmieydbebr5vkok5r34ya464gqz6c44bnyd.onion/p/sponsor96-next-report)
- [x] Get narrative into the report
- [x] Gather indicators
- [x] Send final docs to team for review
- [x] Send final docs to Bekeela and Isabela for review
101
- [x] Send mail [requesting info](https://pad.riseup.net/p/sponsor101-next-report)
- [x] Get narrative into the report
- [x] Gather indicators
- [x] Send final docs to team for review
- [x] Send final docs to Bekeela and Isabela for review
112
- [x] Send mail [requesting info](https://pad.riseup.net/p/sponsor112-next-report)
- [x] Get narrative into the report
- [x] Gather indicators
- [x] Send final docs to team for review
- [x] Send final docs to Bekeela and Isabela for reviewGabagaba@torproject.orgGabagaba@torproject.org2023-10-25https://gitlab.torproject.org/tpo/tpa/team/-/issues/41336mandos not working for dal-rescue-022023-10-02T19:48:53ZJérôme Charaouilavamind@torproject.orgmandos not working for dal-rescue-02When rebooting `dal-rescue-02`, it gets stuck in the initramfs, at the luks prompt.
Normally, mandos takes care of supplying the password to allow the boot process to complete, but it's not working.
I checked the configuration on the m...When rebooting `dal-rescue-02`, it gets stuck in the initramfs, at the luks prompt.
Normally, mandos takes care of supplying the password to allow the boot process to complete, but it's not working.
I checked the configuration on the mandos server and at a glance it checks out. The problem is probably that the luks password in mandos is not present as a luks key slot on `dal-rescue-02`.anarcatanarcathttps://gitlab.torproject.org/tpo/core/arti/-/issues/1050Calculate the worst_case_end time for desc publication attempts2023-10-04T18:02:15Zgabi-250Calculate the worst_case_end time for desc publication attemptsCalculate the `worst_case_end` (the time the publication attempt will definitely be complete or abandoned) and pass call `IptSet::note_publication_attempt` in the publisher.Calculate the `worst_case_end` (the time the publication attempt will definitely be complete or abandoned) and pass call `IptSet::note_publication_attempt` in the publisher.gabi-250gabi-250https://gitlab.torproject.org/tpo/network-health/metrics/website/-/issues/40100Website graphs are not getting updated past 09/162024-01-25T14:28:26ZGeorg KoppenWebsite graphs are not getting updated past 09/16We still seem to have issues getting recent network data displayed on our website. E.g. take https://metrics.torproject.org/userstats-relay-country.html?start=2023-09-01&end=2023-09-26&country=all&events=off:
![userstats-relay-country-al...We still seem to have issues getting recent network data displayed on our website. E.g. take https://metrics.torproject.org/userstats-relay-country.html?start=2023-09-01&end=2023-09-26&country=all&events=off:
![userstats-relay-country-all-2023-09-01-2023-09-26-off](/uploads/085d9f3cfc3352f8b53a983b0c1861fc/userstats-relay-country-all-2023-09-01-2023-09-26-off.png)
This shows data until (and including) 09/16/2023. However, when writing this we already have 09/26/2023. While it can happen that our graphs lack 1 or 2 days, 10 days is not expected.
@hiro: I am filing this ticket for visibility issues, so other folks know we are aware of it. And we can note down what we tried and believed to be the issue and what the fix was.HiroHirohttps://gitlab.torproject.org/tpo/network-health/team/-/issues/330New round of contacting operators for DNS issues and badexiting problematic r...2023-10-03T09:07:36ZGeorg KoppenNew round of contacting operators for DNS issues and badexiting problematic relays (09/25/2023)We got a new report on Monday:
```
Relay 17F41F8DAFA4B36AAB10E202ABA14601AAE1D616 failed DNS check 5/5 times
Relay 359C5231AC2452D365B64A23C27817A1DFEE56B4 failed DNS check 5/5 times
Relay 4C209C991956896A830890ED74A8AE1207EB8AF4 failed ...We got a new report on Monday:
```
Relay 17F41F8DAFA4B36AAB10E202ABA14601AAE1D616 failed DNS check 5/5 times
Relay 359C5231AC2452D365B64A23C27817A1DFEE56B4 failed DNS check 5/5 times
Relay 4C209C991956896A830890ED74A8AE1207EB8AF4 failed DNS check 5/5 times
Relay 5ECD28C3476E6B3BFFC68E3AB9F2DAFBE3238A95 failed DNS check 5/5 times
Relay 89940F610EFB0ED4E624838EAE561ADE55C03321 failed DNS check 5/5 times
Relay B6A5986F404B2C5EB604A37276C0CB7B24FB6631 failed DNS check 5/5 times
Relay CFF9C18036D401579C473177C0D95B463AD371F7 failed DNS check 5/5 times
Relay E0C90700FE1F81044A086591DCB14F02797AC14B failed DNS check 5/5 times
Relay E2C0AD7114510F21B9F09E7900185D440C20CC0E failed DNS check 5/5 times
```
after the one the one from the previous week:
```
Relay 17F41F8DAFA4B36AAB10E202ABA14601AAE1D616 failed DNS check 5/5 times
Relay 2892073608985977DED33F98A9FA27A9C47C8B61 failed DNS check 3/3 times
Relay 359C5231AC2452D365B64A23C27817A1DFEE56B4 failed DNS check 4/4 times
Relay 4C209C991956896A830890ED74A8AE1207EB8AF4 failed DNS check 5/5 times
Relay 585C855F2CA868B2EBFF187CEA1B85508403E86F failed DNS check 5/5 times
Relay 5ECD28C3476E6B3BFFC68E3AB9F2DAFBE3238A95 failed DNS check 5/5 times
Relay 81EDFBC8F6F5C7CF0ADD5F8E08BC8FABA04089C6 failed DNS check 5/5 times
Relay 89940F610EFB0ED4E624838EAE561ADE55C03321 failed DNS check 5/5 times
Relay B6A5986F404B2C5EB604A37276C0CB7B24FB6631 failed DNS check 5/5 times
Relay CFF9C18036D401579C473177C0D95B463AD371F7 failed DNS check 5/5 times
Relay E0C90700FE1F81044A086591DCB14F02797AC14B failed DNS check 5/5 times
Relay E2C0AD7114510F21B9F09E7900185D440C20CC0E failed DNS check 5/5 times
```
So, there are some ongoing problems at a bunch of relays and we should reach out to the operators to get that fixed, if possible.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/community/hackweek/-/issues/16Collaborative editing2023-11-30T16:16:40ZSilvio RhattoCollaborative editing# About the project
* Contact: @rhatto
* Chat: #tor-dev on `irc.oftc.net`
* Video room: to be defined.
# Participants
- @meskio
- @rhatto (I'm looking for someone to be the new maintainer/coordinator/asignee for this proposal, as I ma...# About the project
* Contact: @rhatto
* Chat: #tor-dev on `irc.oftc.net`
* Video room: to be defined.
# Participants
- @meskio
- @rhatto (I'm looking for someone to be the new maintainer/coordinator/asignee for this proposal, as I may participate only marginally on it)
- @micah
- @shelikhoo (Partially)
# Summary
This is about enhancing ways we deal with [Etherpads](https://etherpad.org/).
## Project A - Nextcloud collaborative editor
* [x] Try the Nextcloud collaborative editor as an attempt at reducing the number of external tools we have to deal with.
## Project B - Etherpad
* [x] Etherpad archival utility/GitLab bot (as ticket comments, merge requests etc). Use case: add pad links into the ticket description, and the bot will act regularly, submitting changes somewhere. Maybe this already exists? We may also try to expand this use case for other document sources/platforms.
* [ ] Investigate the possibility to run our own etherpad, and not use the pad cleanup that Riseup does, and then the pads can be integrated with Nextcloud.
## Project C - CryptPad
* [x] Etherpad may be winding down development, and we should re-consider [CryptPad](https://cryptpad.fr/), which has a number of document management utilities bundled with it.
## Project D - HedgeDoc
* [ ] Try [HedgeDoc](https://hedgedoc.org/), "an open-source, web-based, self-hosted, collaborative markdown editor".
# Skills
What are the skills needed for the project:
* Not sure yet! Maybe no specific skills needed.
* Etherpad autosaving may depend on the knowledge needed to create GitLab bots and other scripts.
# LinksHackweek 2023micahmicah@torproject.orgmicahmicah@torproject.org2023-11-09https://gitlab.torproject.org/tpo/tpa/team/-/issues/41334Endpoint for containers.torproject.org is confused2023-09-25T16:22:02Zmicahmicah@torproject.orgEndpoint for containers.torproject.org is confusedIf I attempt to pull an image from the container registry like this:
```
$ podman pull containers.torproject.org/tpo/tpa/container-images:bookworm
```
I get this error:
```
Error: initializing source docker://containers.torproject.org...If I attempt to pull an image from the container registry like this:
```
$ podman pull containers.torproject.org/tpo/tpa/container-images:bookworm
```
I get this error:
```
Error: initializing source docker://containers.torproject.org/tpo/tpa/container-images:bookworm: pinging container registry containers.torproject.org: Get "https://containers.torproject.org/v2/": x509: certificate is valid for gitlab-02.torproject.org, not containers.torproject.org
```Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/web/donate/-/issues/8donate-api is (was) down due to bad htaccess2023-09-28T10:03:07ZKezdonate-api is (was) down due to bad htaccesstoday i received an email from stripe letting me know that over 700 webhook requests to the donate middleware failed. after a small but of investigation, i discovered all endpoints return a 404. this was caused by commit 7a2e74f172fe0f36...today i received an email from stripe letting me know that over 700 webhook requests to the donate middleware failed. after a small but of investigation, i discovered all endpoints return a 404. this was caused by commit 7a2e74f172fe0f36196111cd021c959d58d10453 (!6), which redirected the donate-api index page to donate.tpo, but apparently also broke all other routes.
i've reverted the change on crm-ext-01, and i'll be working to find a better solution for #7https://gitlab.torproject.org/tpo/tpa/team/-/issues/41333Update torbrowser@torproject.org key in wkd2023-09-25T20:46:47ZboklmUpdate torbrowser@torproject.org key in wkdIt is possible to get the torbrowser gpg key with:
```
gpg --auto-key-locate nodefault,wkd --locate-keys torbrowser@torproject.org
```
The key that is returned by this needs to be updated for a new expiration date on the subkey. The upd...It is possible to get the torbrowser gpg key with:
```
gpg --auto-key-locate nodefault,wkd --locate-keys torbrowser@torproject.org
```
The key that is returned by this needs to be updated for a new expiration date on the subkey. The updated key is:
https://people.torproject.org/~boklm/tmp/_torbrowser_extended_2023.ascanarcatanarcathttps://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/issues/49[NEW] Arti CLI Reference Manual2023-12-20T13:42:28Zharleta[NEW] Arti CLI Reference Manualhttps://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/blob/453386dc1a3e83be19bee8218b3e1e76c169aeea/docs/guides/cli-reference.mdhttps://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/blob/453386dc1a3e83be19bee8218b3e1e76c169aeea/docs/guides/cli-reference.mdhttps://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/1Review details endpoint2023-09-22T08:16:21ZHiroReview details endpoint@mattrighetti asked me to give a second review of the details endpoint and on closer look I found a few issues:
on https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/blame/dev/src/metrics/details.rs#L102
I woul...@mattrighetti asked me to give a second review of the details endpoint and on closer look I found a few issues:
on https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/blame/dev/src/metrics/details.rs#L102
I would remove all the joins. I think maybe to provide the network weights we could either query the latest network_status_entry_weights at the time of the server status (https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/blob/main/network_status_tables.sql?ref_type=heads#L70) or make a query to VM proxied, up to the server status time (we have both the weight and the fraction calculated from consensuses https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/blob/main/METRICS.md?ref_type=heads#from-consensuses).
For both relay and bridge what do we need from the server descriptor? I could add that directly into the status so we don't have to make a join query.Mattia RighettiMattia Righettihttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42121Remove "Open in app" feature2023-10-04T18:45:01ZclairehurstRemove "Open in app" feature<!--
* Use this issue template for reporting a new bug.
-->
### Summary
**Summarize the bug encountered concisely.**
There is a feature to open the current site in the relevant already installed app (i.e. youtube.com -> youtube app) whi...<!--
* Use this issue template for reporting a new bug.
-->
### Summary
**Summarize the bug encountered concisely.**
There is a feature to open the current site in the relevant already installed app (i.e. youtube.com -> youtube app) which is a likability issue, potentially de-anonymizing the user by associating the tor session with the new clear net one. This feature should be removed.
### Steps to reproduce:
**How one can reproduce the issue - this is very important.**
1. Open a site (e.g. youtube.com)
2. Go to settings
3. Tap on "Open in app"
4. The app should now open with the current site loaded
### What is the current bug behavior?
**What actually happens.**
The site is loaded into the installed app, potentially de-anonymizing the user
### What is the expected behavior?
**What you want to see instead**
The "Open in app" option removed
### Environment
**Which operating system are you using? For example: Debian GNU/Linux 10.1, Windows 10, Ubuntu Xenial, FreeBSD 12.2, etc.**
**Which installation method did you use? Distribution package (apt, pkg, homebrew), from source tarball, from Git, etc.**
macOS Ventura 13.5.2, Pixel 7 pro API 34 emulator, installed from source code via git
### Relevant logs and/or screenshots
![Screenshot_2023-09-21_at_2.47.38_PM](/uploads/35312b271ad52e59a1f20525c558285a/Screenshot_2023-09-21_at_2.47.38_PM.png){width=25%}
![Screenshot_2023-09-21_at_2.47.59_PM](/uploads/dda169366dcc590f0ff6f46caaefbdb5/Screenshot_2023-09-21_at_2.47.59_PM.png){width=25%}![Screenshot_2023-09-21_at_3.24.31_PM](/uploads/04097c0f8e9c0656eb5357f8936366fb/Screenshot_2023-09-21_at_3.24.31_PM.png)clairehurstclairehursthttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40960Prepare Mullvad Brower Alpha 13.0a62023-10-03T15:01:01ZrichardPrepare Mullvad Brower Alpha 13.0a6<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a mullvad-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a mullvad-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building mullvad-browser tags, labels, etc
- **example** : `91.6.0`
- `$(MULLVAD_BROWSER_MAJOR)` : the Mullvad Browser major version
- **example** : `11`
- `$(MULLVAD_BROWSER_MINOR)` : the Mullvad Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(MULLVAD_BROWSER_VERSION)` : the Mullvad Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(MULLVAD_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(MULLVAD_BROWSER_BUILD_N)` : the mullvad-browser build revision for a given Mullvad Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(MULLVAD_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For **example** :
- if we have multiple Mullvad Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(MULLVAD_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(MULLVAD_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `mullvad-browser`, the `$(MULLVAD_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(MULLVAD_BROWSER_VERSION)` : the published Mullvad Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(MB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Mullvad Browser version
- **example** : `mb-12.0.7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` alpha rebase and security backport tasks have been completed
**NOTE** This can/is often done in conjunction with the equivalent Tor Browser release prep issue
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Mullvad Browser Alpha (and Nightly) are on the `main` branch
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(MULLVAD_BROWSER_BUILD_N)`
- [x] `var/torbrowser_incremental_from` : update to previous Desktop version
- **NOTE**: We try to build incrementals for the previous 3 desktop versions except in the case of a watershed update
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make mullvadbrowser-incrementals-*` step will fail
- [x] Update build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `mullvad-browser` tag
- [ ] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-alpha` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [ ] `steps/mullvad-browser/git_hash` : update with `HEAD` commit of project's `mullvad-browser` branch
- [ ] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for uBlock-origin updates here : https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/
- [x] ***(Optional)*** If new version available, update `ublock-origin` section of `input_files` in `projects/browser/config`
- [x] `URL`
- [x] `sha256sum`
- [x] Check for Mullvad Privacy Companion updates here : https://github.com/mullvad/browser-extension/releases
- [ ] ***(Optional)*** If new version available, update `mullvad-extension` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Update `ChangeLog-MB.txt`
- [ ] Ensure ChangeLog-MB.txt is sync'd between alpha and stable branches
- [ ] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [ ] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [ ] Copy the output of the script to the beginning of `ChangeLog-MB.txt` and update its output
- [ ] Version
- [ ] Browser Name
- [ ] Release Date
- [ ] Under `All Platforms` include any version updates for:
- NoScript
- uBlock-origin
- Mullvad Browser Extension
- Firefox
- [x] Open MR with above changes
- [x] Build the MR after initial review on at least two of:
- [x] Tor Project build machine
- [ ] Mullvad build machine
- [x] Local developer machine
- [x] Ensure builders have matching builds
- [x] Merge
- [x] Sign+Tag
- **NOTE** this must be done by one of:
- boklm
- dan
- ma1
- pierov
- richard
- [x] Run: `make mullvadbrowser-signtag-alpha`
- [x] Push tag to `origin`
</details>
<details>
<summary>QA</summary>
### send the build
- [x] Email Mullvad QA: support@mullvad.net, rui@mullvad.net
<details>
<summary>email template</summary>
Subject:
New build: Mullvad Browser $(MULLVAD_BROWSER_VERION) (unsigned)
Body:
unsigned builds: https://tb-build-05.torproject.org/~$(BUILDER)/builds/mullvadbrowser/alpha/unsigned/$(MB_BUILD_TAG)
changelog:
...
</details>
- ***(Optional)*** Add additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
</details>
<details>
<summary>Signing</summary>
### signing
- [x] Assign this issue to the signer, one of:
- boklm
- richard
- [x] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a mullvad notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/mullvad-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : mullvad browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.mullvadbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [x] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component dist.torproject.org`
- [x] Remove old release data from `/srv/dist-master.torproject.org/htdocs/mullvadbrowser`
- [x] Static update components (again) : `static-update-component dist.torproject.org`
</details>
<details>
<summary>Publishing</summary>
### mullvad-browser (github): https://github.com/mullvad/mullvad-browser/
- [x] Assign this issue to someone with mullvad commit access, one of:
- richard
- [x] Push this release's associated `mullvad-browser.git` branch to github
- [x] Push this release's associated tags to github:
- [ ] Firefox ESR tag
- **example** : `FIREFOX_102_12_0esr_BUILD1,`
- [ ] `base-browser` tag
- **example** : `base-browser-102.12.0esr-12.0-1-build1`
- [x] `mullvad-browser` tag
- **example** : `mullvad-browser-102.12.0esr-12.0-1-build1`
- [x] Sign+Tag additionally the `mullvad-browser.git` `firefox` commit used in build:
- **Tag**: `$(MULLVAD_BROWSER_VERSION)`
- **example** : `12.5a7`
- **Message**: `$(ESR_VERSION)esr-based $(MULLVAD_BROWSER_VERSION)`
- **example** : `102.12.0esr-based 12.5a7`
- [x] Push tag to github
### email
- [x] Email Mullvad with release information: support@mullvad.net, rui@mullvad.net
<details>
<summary>email template</summary>
Subject:
New build: Mullvad Browser $(MULLVAD_BROWSER_VERION) (signed)
Body:
signed builds: https://dist.torproject.org/mullvadbrowser/$(MULLVAD_BROWSER_VERSION)
update_response hashes: $(MULLVAD_UPDATE_RESPONSES_HASH)
changelog:
...
</details>
</details>
<details>
<summary>Downstream</summary>
### notify packagers
- [ ] **(Optional, Once Mullvad Updates their Github Releases Page)** Email downstream consumers:
- **NOTE**: This is an optional step and only necessary close a major release/transition from alpha to stable, or if there are major packing changes these developers need to be aware of
<details>
<summary>email template</summary>
Hello!
Mullvad-Browser $(MULLVAD_BROWSER_VERSION) packages are available, so you should all update your respective downstream packages.
Release builds can be found here:
- https://github.com/mullvad/mullvad-browser/releases/tag/$(MULLVAD_BROWSER_VERSION)
</details>
- flathub package maintainer: proletarius101@protonmail.com
- arch package maintainer: bootctl@gmail.com
- nixOS package maintainer: dev@felschr.com
</details>richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40959Prepare Tor Browser Alpha 13.0a62023-10-03T15:01:04ZrichardPrepare Tor Browser Alpha 13.0a6<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** :...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building tor-browser tags, labels, etc
- **example** : `91.6.0`
- `$(TOR_BROWSER_MAJOR)` : the Tor Browser major version
- **example** : `11`
- `$(TOR_BROWSER_MINOR)` : the Tor Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(TOR_BROWSER_VERSION)` : the Tor Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(TOR_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(TOR_BROWSER_BUILD_N)` : the tor-browser build revision for a given Tor Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(TOR_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For example :
- if we have multiple Tor Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(TOR_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(TOR_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `tor-browser`, the `$(TOR_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(TOR_BROWSER_VERSION)` : the published Tor Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(TBB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Tor Browser version
- **example** : `tbb-12.5a7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` stable rebase and security backport tasks have been completed
**NOTE** This can/is often done in conjunction with the equivalent Mullvad Browser release prep issue
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Tor Browser Alpha (and Nightly) are on the `main` branch
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(TOR_BROWSER_BUILD_N)`
- [x] ***(Desktop Only)***`var/torbrowser_incremental_from` : update to previous Desktop version
- **NOTE**: We try to build incrementals for the previous 3 desktop versions except in the case of a watershed update
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make torbrowser-incrementals-*` step will fail
- [x] Update Desktop-specific build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-alpha` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [x] `steps/tor-browser/git_hash` : update with `HEAD` commit of project's `tor-browser` branch
- [x] `steps/fenix/git_hash` : update with `HEAD` commit of project's `fenix-torbrowserstringsxml` branch
- [x] Update Android-specific build configs
- [x] Update `projects/geckoview/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/geckoview_version` : update to latest `$(ESR_VERSION)` if rebased
- [ ] ***(Optional)*** Update `projects/tor-android-service/config`
- [ ] `git_hash` : update with `HEAD` commit of project's `main` branch
- [ ] ***(Optional)*** Update `projects/application-services/config`:
**NOTE** we don't currently have any of our own patches for this project
- [ ] `git_hash` : update to appropriate git commit associated with `$(ESR_VERSION)`
- [x] ***(Optional)*** Update `projects/firefox-android/config`:
- [x] `fenix_version` : update to match alpha `firefox-android` build tag
- [ ] `browser_branch` : update to match alpha `firefox-android` build tag
- [x] Update allowed_addons.json by running (from `tor-browser-build` root):
- `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
- [x] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for OpenSSL updates here : https://www.openssl.org/source/
- [ ] ***(Optional)*** If new 3.0.X version available, update `projects/openssl/config`
- [ ] `version` : update to next 3.0.X version
- [ ] `input_files/sha256sum` : update to sha256 sum of source tarball
- [x] Check for zlib updates here: https://github.com/madler/zlib/releases
- [ ] **(Optional)** If new tag available, update `projects/zlib/config`
- [ ] `version` : update to next release tag
- [x] Check for tor updates here : https://gitlab.torproject.org/tpo/core/tor/-/tags
- [x] ***(Optional)*** Update `projects/tor/config`
- [x] `version` : update to latest `-alpha` tag or release tag if newer (ping dgoulet or ahf if unsure)
- [x] Check for go updates here : https://golang.org/dl
- **NOTE** : Tor Browser Alpha uses the latest Stable major series go version
- [ ] ***(Optional)*** Update `projects/go/config`
- [ ] `version` : update go version
- [ ] `input_files/sha256sum` for `go` : update sha256sum of archive (sha256 sums are displayed on the go download page)
- [x] Check for manual updates by running (from `tor-browser-build` root): `./tools/fetch-manual.py`
- [ ] ***(Optional)*** If new version is available:
- [ ] Upload the downloaded `manual_$PIPELINEID.zip` file to people.tpo
- [ ] Update `projects/manual/config`:
- [ ] Change the `version` to `$PIPELINEID`
- [ ] Update `sha256sum` in the `input_files` section
- [ ] ***(Optional)*** Update the URL if you have uploaded to a different people.tpo home
- [x] Update `ChangeLog-TBB.txt`
- [x] Ensure ChangeLog-TBB.txt is sync'd between alpha and stable branches
- [ ] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [ ] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [ ] Copy the output of the script to the beginning of `ChangeLog-TBB.txt` and update its output
- [ ] Version
- [ ] Browser Name
- [ ] Release Date
- [ ] Under `All Platforms` include any version updates for:
- NoScript
- tor
- OpenSSL
- lyrebird
- Snowflake
- [ ] Under `Windows + macOS + Linux` include any version updates for:
- Firefox
- [ ] Under `Android` include any version updates for:
- Geckoview
- [ ] Under `Windows + Android` include any version updates for:
- zlib
- [ ] Under `Build System/All Platforms` include any version updates for:
- Go
- [x] Open MR with above changes
- [x] Build the MR after initial review on at least two of:
- [x] Tor Project build machine
- [ ] Mullvad build machine
- [x] Local developer machine
- [x] Ensure builders have matching builds
- [x] Merge
- [x] Sign_Tag
- **NOTE** this must be done by one of:
- boklm
- dan
- ma1
- pierov
- richard
- [x] Run: `make torbrowser-signtag-alpha`
- [x] Push tag to `origin`
</details>
<details>
<summary>Communications</summary>
### notify stakeholders
- [x] Email tor-qa mailing list: tor-qa@lists.torproject.org
<details>
<summary>email template</summary>
Subject:
Tor Browser $(TOR_BROWSER_VERION) (Android, Windows, macOS, Linux)
Body:
Hello All,
Unsigned Tor Browser $(TOR_BROWSER_VERSION) alpha candidate builds are now available for testing:
- https://tb-build-05.torproject.org/~$(BUILDER)/builds/alpha/unsigned/$(TOR_BROWSER_VERSION)/
The full changelog can be found here:
- https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/raw/$(TBB_BUILD_TAG)/projects/browser/Bundle-Data/Docs-TBB/ChangeLog.txt
</details>
- ***(Optional)*** Additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
- [x] ***(Optional, only around build/packaging changes)*** Email packagers:
- Recipients:
- Tails dev mailing list: tails-dev@boum.org
- Guardian Project: nathan@guardianproject.info
- torbrowser-launcher: micah@micahflee.com
- FreeBSD port: freebsd@sysctl.cz <!-- Gitlab user maxfx -->
- OpenBSD port: caspar@schutijser.com <!-- Gitlab user cschutijser -->
- [ ] Note any changes which may affect packaging/downstream integration
- [ ] Email external partners:
- ***(Optional, after ESR migration)*** Cloudflare: ask-research@cloudflare.com
- **NOTE** : We need to provide them with updated user agent string so they can update their internal machinery to prevent Tor Browser users from getting so many CAPTCHAs
</details>
<details>
<summary>Signing</summary>
### signing
- **NOTE** : In practice, it's most efficient to have the blog post and website updates ready to merge, since signing doesn't take very long
- [x] Assign this issue to the signer, one of:
- boklm
- richard
- [ ] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build` is on the right commit: `git tag -v tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N) && git checkout tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N)`
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a tor notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/tor-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : tor browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.torbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [ ] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Enable update responses : `sudo -u tb-release ./deploy_update_responses-alpha.sh`
- [x] Remove old release data from following places:
- **NOTE** : Skip this step if we need to hold on to older versions for some reason (for example, this is an Andoid or Desktop-only release, or if we need to hold back installers in favor of build-to-build updates if there are signing issues, etc)
- [x] `/srv/cdn-master.torproject.org/htdocs/aus1/torbrowser`
- [x] `/srv/dist-master.torproject.org/htdocs/torbrowser`
- [x] Static update components (again) : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Publish APKs to Google Play:
- Log into https://play.google.com/apps/publish
- Select `Tor Browser (Alpha)` app
- Navigate to `Release > Production` and click `Create new release` button:
- Upload the `*.multi.apk` APKs
- Update Release Name to Tor Browser version number
- Update Release Notes
- Next to 'Release notes', click `Copy from a previous release`
- Edit blog post url to point to most recent blog post
- Save, review, and configure rollout percentage
- [ ] 25% rollout when publishing a scheduled update
- [x] 100% rollout when publishing a security-driven release
- [ ] Update rollout percentage to 100% after confirmed no major issues
</details>
<details>
<summary>Signature verification</summary>
<details>
<summary>Check whether the .exe files got properly signed and timestamped</summary>
```bash
# Point OSSLSIGNCODE to your osslsigncode binary
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
OSSLSIGNCODE=/path/to/osslsigncode
../../../tools/authenticode_check.sh
popd
```
</details>
<details>
<summary>Check whether the MAR files got properly signed</summary>
```bash
# Point NSSDB to your nssdb containing the mar signing certificate
# Point SIGNMAR to your signmar binary
# Point LD_LIBRARY_PATH to your mar-tools directory
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
NSSDB=/path/to/nssdb
SIGNMAR=/path/to/mar-tools/signmar
LD_LIBRARY_PATH=/path/to/mar-tools/
../../../tools/marsigning_check.sh
popd
```
</details>
</details>
<details>
<summary>Publishing</summary>
### website: https://gitlab.torproject.org/tpo/web/tpo.git
- [x] `databags/versions.ini` : Update the downloads versions
- `torbrowser-stable/version` : sort of a catch-all for latest stable version
- `torbrowser-alpha/version` : sort of a catch-all for latest stable version
- `torbrowser-*-stable/version` : platform-specific stable versions
- `torbrowser-*-alpha/version` : platform-specific alpha versions
- `tor-stable`,`tor-alpha` : set by tor devs, do not touch
- [x] Push to origin as new branch, open 'Draft :' MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and builds are published
### blog: https://gitlab.torproject.org/tpo/web/blog.git
- [x] Duplicate previous Stable or Alpha release blog post as appropriate to new directory under `content/blog/new-release-tor-browser-$(TOR_BROWSER_VERSION)` and update with info on release :
- [ ] Run `tools/signing/create-blog-post` which should create the new blog post from a template (edit set-config.blog to set you local blog directory)
- [ ] Update Tor Browser version numbers
- [ ] Note any ESR rebase
- [ ] Link to any Firefox security updates from ESR upgrade
- [ ] Link to any Android-specific security backports
- [ ] Note any updates to :
- tor
- OpenSSL
- NoScript
- [ ] Convert ChangeLog-TBB.txt to markdown format used here by :
- `tor-browser-build/tools/changelog-format-blog-post`
- [x] Push to origin as new branch, open `Draft:` MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and website has been updated
### tor-announce mailing list
- [x] Email tor-announce mailing list: tor-announce@lists.torproject.org
<details>
<summary>email template</summary>
Subject:
New Release: Tor Browser $(TOR_BROWSER_VERSION) (Android, Windows, macOS, Linux)
Body:
Hi everyone,
Tor Browser $(TOR_BROWSER_VERSION) has now been published for all platforms. For details please see our blog post:
- $(BLOG_POST_URL)
</details>
- **(Optional)** Additional information:
- [ ] Link to any known issues
</details>richardrichard