The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2023-10-03T13:28:14Zhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41991Review Mozilla 1812518: Allow a custom View for 3rd party downloads2023-10-03T13:28:14ZrichardReview Mozilla 1812518: Allow a custom View for 3rd party downloadsLink: https://bugzilla.mozilla.org/show_bug.cgi?id=1812518
Unclear what 3rd party downloads or how they are meant to work, but we def don't want files downloaded via tor browser android to be re-routed outside the browser and not throug...Link: https://bugzilla.mozilla.org/show_bug.cgi?id=1812518
Unclear what 3rd party downloads or how they are meant to work, but we def don't want files downloaded via tor browser android to be re-routed outside the browser and not through tor.Dan BallardDan Ballardhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41295Intermittant docker failure inside of runners2023-09-21T16:12:04Zmicahmicah@torproject.orgIntermittant docker failure inside of runnersPeriodically, and intermittently, the following failure happens with CI runner jobs:
`ERROR: Job failed (system failure): prepare environment: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon runn...Periodically, and intermittently, the following failure happens with CI runner jobs:
`ERROR: Job failed (system failure): prepare environment: Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running? (docker.go:570:120s). Check https://docs.gitlab.com/runner/shells/index.html#shell-profile-loading for more information`
Here is [an example pipeline](https://gitlab.torproject.org/Diziet/arti/-/jobs/337208), restarting the job will sometimes "fix" it.
My understanding of this failure is that the runner is attempting to use the docker socket but is not able to actually access the socket inside the executor. The solutions for this are to map the docker socket into the runner, for example, the following (limited) snippet from a gitlab-runner `config.toml`:
```
[runners.docker]
...
volumes = ["/certs/client", "/cache", "/run/docker.sock:/run/docker.sock"]
...
```
So first possibility is the volume configuration, where you specify how the docker socket is mapped to the array of volumes.
Second possibility is using DiND (docker-in-docker), with a privileged container, which does not require you to map the docker socket, but does require you to have the option `privileged = true`. I believe that TPA has decided not to allow DiND in the past.
A couple other possibilities that should be also on the table:
- make sure that the gitlab-runner package is up to date. Older versions have issues, use the gitlab 3rd party debian packages for this
- make sure the docker packages are up to date, use the ones provided by *upstream* in their 3rd party repo, as the ones in Debian have these issues
- make sure the OS is running at *least* bullseye
- restart the gitlab-runner and the docker-runner - this shouldn't be necessary, but I've seen this get wedged in irritating ways before (but it stopped once I started using the upstream docker versions, and had the configuration settings right
Checklist:
- [x] try podman as a gitlab runner to see if it has the same problems (it doesn't!)
- [x] run podman for a while to shake out problems (we found a few, but all should be fixed now)
- [x] setup a large runner on ci-runner-x86-02
- [x] pause then retire ci-runner-x86-01
- [x] setup podman runners on chi-node-14
- [x] purge docker from chi-node-14Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41984Rename languageNotification.ftl to base-browser.ftl2023-10-03T15:38:00ZhenryRename languageNotification.ftl to base-browser.ftlWe want to have two main files for our tor browser strings: base-browser.ftl and tor-browser.ftl.
The current languageNotification.ftl file is only used in base-browser, so we can just rename the file, rather than migrate the strings.
...We want to have two main files for our tor browser strings: base-browser.ftl and tor-browser.ftl.
The current languageNotification.ftl file is only used in base-browser, so we can just rename the file, rather than migrate the strings.
CC @pierov and @emmapeelhenryhenryhttps://gitlab.torproject.org/tpo/ux/design/-/issues/57Expand the number of favicons available in our icon libraries2023-08-10T18:12:18ZdonutsExpand the number of favicons available in our icon librariesAt the moment, we have favicons for the following external sites in our icon library:
- propublica.org
- securedrop.org
We should expand this set to include other sites we commonly visualize, e.g. torproject.org and the New York Times.At the moment, we have favicons for the following external sites in our icon library:
- propublica.org
- securedrop.org
We should expand this set to include other sites we commonly visualize, e.g. torproject.org and the New York Times.Jag TalonJag Talonhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41294RT spam-filtered an important email2023-08-29T15:33:28ZmattlavRT spam-filtered an important emailThis person is displeased because he thinks he canceled his large monthly donation, but got charged again:
https://rt.torproject.org/Ticket/Display.html?id=312946
But there isn't any ticket in which he made such a request - and the pre...This person is displeased because he thinks he canceled his large monthly donation, but got charged again:
https://rt.torproject.org/Ticket/Display.html?id=312946
But there isn't any ticket in which he made such a request - and the present ticket got moved to the spam queue:
`Ticket did not pass DMARC verification, moved to spam queue.`
It's inconvenient to miss an email such as this. I assume this occurred because this person's email setup is messed up, but I also have to ask if our spam filter is tuned correctly, or if this message - and probably its predecessor - should have come through?Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41980Circuit display headline is misaligned in 13.0 alpha2023-10-03T13:28:19ZdonutsCircuit display headline is misaligned in 13.0 alphaThe headline reading "Circuit display for <website>" is vertically misaligned in 13.0 alpha. See Firefox's "Site information" wingpanel for the correct alignment.The headline reading "Circuit display for <website>" is vertically misaligned in 13.0 alpha. See Firefox's "Site information" wingpanel for the correct alignment.https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41979"Use current bridges" toggle in Connection settings should be a switch in 13....2023-10-04T15:53:11Zdonuts"Use current bridges" toggle in Connection settings should be a switch in 13.0 alphaFor some reason, the "Use current bridges" toggle appears to have swapped from a switch to a checkbox in 13.0 alpha. See this issue for a screenshot of how it looks in 12.5 stable instead: https://gitlab.torproject.org/tpo/applications/t...For some reason, the "Use current bridges" toggle appears to have swapped from a switch to a checkbox in 13.0 alpha. See this issue for a screenshot of how it looks in 12.5 stable instead: https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41912https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41977Hide the "Learn more" link in bridge cards2023-10-03T13:28:21ZdonutsHide the "Learn more" link in bridge cardsBoth the "What are these?" and "Learn more" links point at the same manual link: `about:manual#bridges_bridge-moji`
Given that, let's remove the "Learn more" link since the former is more contextual please.Both the "What are these?" and "Learn more" links point at the same manual link: `about:manual#bridges_bridge-moji`
Given that, let's remove the "Learn more" link since the former is more contextual please.richardrichardhttps://gitlab.torproject.org/tpo/core/arti/-/issues/1000TROVE-2023-002 bridges.enabled=true can be accepted if bridge support compile...2023-08-08T17:54:50ZIan Jacksoniwj@torproject.orgTROVE-2023-002 bridges.enabled=true can be accepted if bridge support compiled out### Steps to reproduce
```
cargo build --no-default-features --features=native-tls,tokio -p arti
```
Make `~/.config/arti/arti.toml` containing
```
[bridges]
enabled = true
```
(but no actual bridges)
Run `target/debug/arti proxy`
##...### Steps to reproduce
```
cargo build --no-default-features --features=native-tls,tokio -p arti
```
Make `~/.config/arti/arti.toml` containing
```
[bridges]
enabled = true
```
(but no actual bridges)
Run `target/debug/arti proxy`
### Expected behaviour
Arti refuses to start, saying:
`target/debug/arti: error: read configuration: Config semantically incorrect: Fields ["bridges.enabled", "bridges.bridges"] are inconsistent: bridges enabled=true, but no bridges defined`
### Actual behaviour
Arti runs, and connects to guards directly.Ian Jacksoniwj@torproject.orgIan Jacksoniwj@torproject.orghttps://gitlab.torproject.org/tpo/community/hackweek/-/issues/10Issue template for Hackweek proposals2023-11-30T16:17:39ZSilvio RhattoIssue template for Hackweek proposalsCreate an issue template for Hackweek proposals, ref. [ORGANIZE-PRE-HACKWEEK](https://gitlab.torproject.org/tpo/community/hackweek/-/blob/main/ORGANIZE-PRE-HACKWEEK.md).Create an issue template for Hackweek proposals, ref. [ORGANIZE-PRE-HACKWEEK](https://gitlab.torproject.org/tpo/community/hackweek/-/blob/main/ORGANIZE-PRE-HACKWEEK.md).Hackweek 2023Gabagaba@torproject.orgGabagaba@torproject.orghttps://gitlab.torproject.org/tpo/network-health/metrics/onionperf/-/issues/40068Link to onionperf documentation HTML version is 4042023-08-24T10:31:27Zcomputer_freakLink to onionperf documentation HTML version is 404The link to the [onionperf documentation](https://gitlab.torproject.org/tpo/network-health/metrics/onionperf#documentation) HTML version first asks me if i authorize GitLab Pages to use my account but once granted it is 404.The link to the [onionperf documentation](https://gitlab.torproject.org/tpo/network-health/metrics/onionperf#documentation) HTML version first asks me if i authorize GitLab Pages to use my account but once granted it is 404.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/network-health/team/-/issues/316Metrics is overwritting userstats-bridge-combined in August 20232024-01-25T14:30:17ZGusMetrics is overwritting userstats-bridge-combined in August 2023While checking the [userstats-bridge-combined](https://metrics.torproject.org/userstats-bridge-combined.html) in some countries, I noticed that the data in August is being overwritten.
I downloaded the csv some days ago and 2023-08-03 d...While checking the [userstats-bridge-combined](https://metrics.torproject.org/userstats-bridge-combined.html) in some countries, I noticed that the data in August is being overwritten.
I downloaded the csv some days ago and 2023-08-03 data was there:
```
2023-07-30,tm,<OR>,2,15,99
2023-07-30,tm,conjure,1,1,99
2023-07-30,tm,meek,738,738,99
2023-07-30,tm,obfs4,5432,5440,99
2023-07-30,tm,snowflake,9,14,99
2023-08-03,tm,<OR>,2,87,98
2023-08-03,tm,conjure,3,3,98
2023-08-03,tm,meek,933,936,98
2023-08-03,tm,obfs4,4391,4468,98
2023-08-03,tm,snowflake,6,12,98
```
But today I downloaded the csv file and only found the data for 2023-08-05:
```
2023-07-28,tm,<OR>,54,72,94
2023-07-28,tm,conjure,0,0,94
2023-07-28,tm,meek,654,655,94
2023-07-28,tm,obfs4,5343,5355,94
2023-07-28,tm,snowflake,13,19,94
2023-07-30,tm,<OR>,2,15,99
2023-07-30,tm,conjure,1,1,99
2023-07-30,tm,meek,738,738,99
2023-07-30,tm,obfs4,5429,5438,99
2023-07-30,tm,snowflake,9,14,99
2023-08-05,tm,<OR>,699,804,98
2023-08-05,tm,conjure,4,4,98
2023-08-05,tm,meek,935,935,98
2023-08-05,tm,obfs4,5159,5259,98
2023-08-05,tm,snowflake,4,9,98
```
https://metrics.torproject.org/userstats-bridge-combined.html?start=2023-05-10&end=2023-08-08&country=tm
(cc @hiro)HiroHirohttps://gitlab.torproject.org/tpo/web/community/-/issues/320Create WebTunnel bridges guide2023-08-09T15:42:01ZGusCreate WebTunnel bridges guideAC-Team wrote a guide how to deploy WebTunnel bridges, and now we need to review the instructions and move to the Community portal.
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/webtunnel/-/blob/main/README.mdAC-Team wrote a guide how to deploy WebTunnel bridges, and now we need to review the instructions and move to the Community portal.
https://gitlab.torproject.org/tpo/anti-censorship/pluggable-transports/webtunnel/-/blob/main/README.mdGusGushttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40915Prepare Mullvad Browser Alpha 13.0a32023-08-25T19:34:24ZrichardPrepare Mullvad Browser Alpha 13.0a3<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a mullvad-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a mullvad-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building mullvad-browser tags, labels, etc
- **example** : `91.6.0`
- `$(MULLVAD_BROWSER_MAJOR)` : the Mullvad Browser major version
- **example** : `11`
- `$(MULLVAD_BROWSER_MINOR)` : the Mullvad Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(MULLVAD_BROWSER_VERSION)` : the Mullvad Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(MULLVAD_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(MULLVAD_BROWSER_BUILD_N)` : the mullvad-browser build revision for a given Mullvad Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(MULLVAD_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For **example** :
- if we have multiple Mullvad Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(MULLVAD_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(MULLVAD_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `mullvad-browser`, the `$(MULLVAD_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(MULLVAD_BROWSER_VERSION)` : the published Mullvad Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(MB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Mullvad Browser version
- **example** : `mb-12.0.7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` alpha rebase and security backport tasks have been completed
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Mullvad Browser Alpha (and Nightly) are on the `main` branch
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(MULLVAD_BROWSER_BUILD_N)`
- [x] `var/torbrowser_incremental_from` : update to previous Desktop version
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make mullvadbrowser-incrementals-*` step will fail
- [x] Update build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `mullvad-browser` tag
- [x] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-alpha` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [x] `steps/mullvad-browser/git_hash` : update with `HEAD` commit of project's `mullvad-browser` branch
`basebrowser-newidentityftl` branch
- [x] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for uBlock-origin updates here : https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/
- [ ] ***(Optional)*** If new version available, update `ublock-origin` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for Mullvad Privacy Companion updates here : https://github.com/mullvad/browser-extension/releases
- [ ] ***(Optional)*** If new version available, update `mullvad-extension` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Open MR with above changes
- [x] Merge
- [x] Sign/Tag commit: `make mullvadbrowser-signtag-alpha`
- [x] Push tag to `origin`
- [x] Begin build on `$(BUILD_SERVER)` (fix any issues in subsequent MRs)
- [ ] **TODO** Submit build-tag to Mullvad build infra
- [x] Ensure builders have matching builds
</details>
<details>
<summary>QA</summary>
### send the build
- [x] Email Mullvad QA: support@mullvad.net, rui@mullvad.net
<details>
<summary>email template</summary>
Subject:
New build: Mullvad Browser $(MULLVAD_BROWSER_VERION) (unsigned)
Body:
unsigned builds: https://tb-build-05.torproject.org/~$(BUILDER)/builds/mullvadbrowser/release/unsigned/$(MB_BUILD_TAG)
changelog:
...
</details>
- ***(Optional)*** Add additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
</details>
<details>
<summary>Signing</summary>
### signing
- [x] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a mullvad notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/mullvad-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : mullvad browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.mullvadbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [x] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component dist.torproject.org`
- [x] Remove old release data from `/srv/dist-master.torproject.org/htdocs/mullvadbrowser`
- [x] Static update components (again) : `static-update-component dist.torproject.org`
</details>
<details>
<summary>Publishing</summary>
### email
- [x] Email Mullvad with release information: support@mullvad.net, rui@mullvad.net
<details>
<summary>email template</summary>
Subject:
New build: Mullvad Browser $(MULLVAD_BROWSER_VERION) (signed)
Body:
signed builds: https://dist.torproject.org/mullvadbrowser/$(MULLVAD_BROWSER_VERSION)
update_response hashes: $(MULLVAD_UPDATE_RESPONSES_HASH)
changelog:
...
</details>
### mullvad-browser (github): https://github.com/mullvad/mullvad-browser/
- [x] Push this release's associated `mullvad-browser.git` branch to github
- [x] Push this release's associated tags to github:
- [x] Firefox ESR tag
- **example** : `FIREFOX_102_12_0esr_BUILD1,`
- [x] `base-browser` tag
- **example** : `base-browser-102.12.0esr-12.0-1-build1`
- [x] `mullvad-browser` tag
- **example** : `mullvad-browser-102.12.0esr-12.0-1-build1`
- [x] Sign+Tag additionally the `mullvad-browser.git` `firefox` commit used in build:
- **Tag**: `$(MULLVAD_BROWSER_VERSION)`
- **example** : `12.5a7`
- **Message**: `$(ESR_VERSION)esr-based $(MULLVAD_BROWSER_VERSION)`
- **example** : `102.12.0esr-based 12.5a7`
- [x] Push tag to github
</details>
<details>
<summary>Downstream</summary>
### notify packagers
- [x] **(Optional, Once Mullvad Updates their Github Releases Page)** Email downstream consumers:
<details>
<summary>email template</summary>
...
...
</details>
- **NOTE**: This is an optional step and only necessary close a major release/transition from alpha to stable, or if there are major packing changes these developers need to be aware of
- [x] flathub package maintainer: proletarius101@protonmail.com
- [x] arch package maintainer: bootctl@gmail.com
- [x] nixOS package maintainer: dev@felschr.com
</details>richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40914Prepare Tor Browser Alpha 13.0a32023-08-24T17:42:16ZrichardPrepare Tor Browser Alpha 13.0a3<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** :...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building tor-browser tags, labels, etc
- **example** : `91.6.0`
- `$(TOR_BROWSER_MAJOR)` : the Tor Browser major version
- **example** : `11`
- `$(TOR_BROWSER_MINOR)` : the Tor Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(TOR_BROWSER_VERSION)` : the Tor Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(TOR_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(TOR_BROWSER_BUILD_N)` : the tor-browser build revision for a given Tor Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(TOR_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For example :
- if we have multiple Tor Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(TOR_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(TOR_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `tor-browser`, the `$(TOR_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(TOR_BROWSER_VERSION)` : the published Tor Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(TBB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Tor Browser version
- **example** : `tbb-12.5a7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` stable rebase and security backport tasks have been completed
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Tor Browser Alpha (and Nightly) are on the `main` branch
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(TOR_BROWSER_BUILD_N)`
- [x] ***(Desktop Only)***`var/torbrowser_incremental_from` : update to previous Desktop version
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make torbrowser-incrementals-*` step will fail
- [x] Update Desktop-specific build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [x] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-alpha` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [x] `steps/tor-browser/git_hash` : update with `HEAD` commit of project's `tor-browser` branch
- [x] `steps/fenix/git_hash` : update with `HEAD` commit of project's `fenix-torbrowserstringsxml` branch
- [x] Update Android-specific build configs
- [x] Update `projects/geckoview/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [x] ***(Optional)*** `var/geckoview_version` : update to latest `$(ESR_VERSION)` if rebased
- [ ] ***(Optional)*** Update `projects/tor-android-service/config`
- [ ] `git_hash` : update with `HEAD` commit of project's `main` branch
- [ ] ***(Optional)*** Update `projects/application-services/config`:
**NOTE** we don't currently have any of our own patches for this project
- [ ] `git_hash` : update to appropriate git commit associated with `$(ESR_VERSION)`
- [ ] ***(Optional)*** Update `projects/firefox-android/config`:
- [ ] `fenix_version` : update to match alpha `firefox-android` build tag
- [ ] `browser_branch` : update to match alpha `firefox-android` build tag
- [x] Update allowed_addons.json by running (from `tor-browser-build` root):
- `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
- [x] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for OpenSSL updates here : https://www.openssl.org/source/
- [ ] ***(Optional)*** If new 1.X.Y version available, update `projects/openssl/config`
- [ ] `version` : update to next 1.X.Y version
- [ ] `input_files/sha256sum` : update to sha256 sum of source tarball
- [x] Check for zlib updates here: https://github.com/madler/zlib/releases
- [ ] **(Optional)** If new tag available, update `projects/zlib/config`
- [ ] `version` : update to next release tag
- [x] Check for tor updates here : https://gitlab.torproject.org/tpo/core/tor/-/tags
- [ ] ***(Optional)*** Update `projects/tor/config`
- [ ] `version` : update to latest `-alpha` tag or release tag if newer (ping dgoulet or ahf if unsure)
- [x] Check for go updates here : https://golang.org/dl
- **NOTE** : Tor Browser Alpha uses the latest Stable major series go version
- [ ] ***(Optional)*** Update `projects/go/config`
- [ ] `version` : update go version
- [ ] `input_files/sha256sum` for `go` : update sha256sum of archive (sha256 sums are displayed on the go download page)
- [x] Check for manual updates by running (from `tor-browser-build` root): `./tools/fetch-manual.py`
- [ ] ***(Optional)*** If new version is available:
- [ ] Upload the downloaded `manual_$PIPELINEID.zip` file to people.tpo
- [ ] Update `projects/manual/config`:
- [ ] Change the `version` to `$PIPELINEID`
- [ ] Update `sha256sum` in the `input_files` section
- [ ] ***(Optional)*** Update the URL if you have uploaded to a different people.tpo home
- [x] Update `ChangeLog.txt`
- [ ] Ensure ChangeLog.txt is sync'd between alpha and stable branches
- [ ] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [ ] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [ ] Copy the output of the script to the beginning of `ChangeLog.txt` and adjust its output
- **NOTE** : If you used the issue number, you will need to write the Tor Browser version manually
- [ ] ***(Optional)*** Under `All Platforms` include any version updates for:
- [ ] Translations
- [ ] OpenSSL
- [ ] NoScript
- [ ] zlib
- [ ] tor daemon
- [ ] ***(Optional)*** Under `Windows + macOS + Linux` include updates for:
- [ ] Firefox
- [ ] ***(Optional)*** Under `Android`, include updates for:
- [ ] Geckoview
- [ ] ***(Optional)*** Under `Build System/All Platforms` include updates for:
- [ ] Go
- [x] Open MR with above changes
- [x] Merge
- [x] Sign/Tag commit: `make torbrowser-signtag-alpha`
- [x] Push tag to `origin`
- [x] Begin build on `$(BUILD_SERVER)` (fix any issues in subsequent MRs)
- [ ] **TODO** Submit build-tag to Mullvad build infra
- [x] Ensure builders have matching builds
</details>
<details>
<summary>Communications</summary>
### notify stakeholders
<details>
<summary>email template</summary>
Subject:
Tor Browser $(TOR_BROWSER_VERION) (Android, Windows, macOS, Linux)
Body:
Hello All,
Unsigned Tor Browser $(TOR_BROWSER_VERSION) alpha candidate builds are now available for testing:
- https://tb-build-05.torproject.org/~$(BUILDER)/builds/release/unsigned/$(TOR_BROWSER_VERSION)/
The full changelog can be found here:
- https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/$(TBB_BUILD_TAG)/ChangeLog.txt
</details>
- [x] Email tor-qa mailing list: tor-qa@lists.torproject.org
- ***(Optional)*** Additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
- [x] ***(Optional, only around build/packaging changes)*** Email packagers:
- Recipients:
- Tails dev mailing list: tails-dev@boum.org
- Guardian Project: nathan@guardianproject.info
- torbrowser-launcher: micah@micahflee.com
- FreeBSD port: freebsd@sysctl.cz <!-- Gitlab user maxfx -->
- OpenBSD port: caspar@schutijser.com <!-- Gitlab user cschutijser -->
- [x] Note any changes which may affect packaging/downstream integration
- [x] Email external partners:
- ***(Optional, after ESR migration)*** Cloudflare: ask-research@cloudflare.com
- **NOTE** : We need to provide them with updated user agent string so they can update their internal machinery to prevent Tor Browser users from getting so many CAPTCHAs
</details>
<details>
<summary>Signing</summary>
### signing
- **NOTE** : In practice, it's most efficient to have the blog post and website updates ready to merge, since signing doesn't take very long
- [ ] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build` is on the right commit: `git tag -v tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N) && git checkout tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N)`
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a tor notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/tor-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : tor browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.torbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [ ] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Enable update responses : `sudo -u tb-release ./deploy_update_responses-alpha.sh`
- [ ] Remove old release data from following places:
- **NOTE** : Skip this step if we need to hold on to older versions for some reason (for example, this is an Andoid or Desktop-only release, or if we need to hold back installers in favor of build-to-build updates if there are signing issues, etc)
- [ ] `/srv/cdn-master.torproject.org/htdocs/aus1/torbrowser`
- [ ] `/srv/dist-master.torproject.org/htdocs/torbrowser`
- [ ] Static update components (again) : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Publish APKs to Google Play:
- Log into https://play.google.com/apps/publish
- Select `Tor Browser (Alpha)` app
- Navigate to `Release > Production` and click `Create new release` button:
- Upload the `*.multi.apk` APKs
- Update Release Name to Tor Browser version number
- Update Release Notes
- Next to 'Release notes', click `Copy from a previous release`
- Edit blog post url to point to most recent blog post
- Save, review, and configure rollout percentage
- [x] 25% rollout when publishing a scheduled update
- [ ] 100% rollout when publishing a security-driven release
- [ ] Update rollout percentage to 100% after confirmed no major issues
</details>
<details>
<summary>Signature verification</summary>
<details>
<summary>Check whether the .exe files got properly signed and timestamped</summary>
```
# Point OSSLSIGNCODE to your osslsigncode binary
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
OSSLSIGNCODE=/path/to/osslsigncode
../../../tools/authenticode_check.sh
popd
```
</details>
<details>
<summary>Check whether the MAR files got properly signed</summary>
```
# Point NSSDB to your nssdb containing the mar signing certificate
# Point SIGNMAR to your signmar binary
# Point LD_LIBRARY_PATH to your mar-tools directory
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
NSSDB=/path/to/nssdb
SIGNMAR=/path/to/mar-tools/signmar
LD_LIBRARY_PATH=/path/to/mar-tools/
../../../tools/marsigning_check.sh
popd
```
</details>
</details>
<details>
<summary>Publishing</summary>
### website: https://gitlab.torproject.org/tpo/web/tpo.git
- [x] `databags/versions.ini` : Update the downloads versions
- `torbrowser-stable/version` : sort of a catch-all for latest stable version
- `torbrowser-alpha/version` : sort of a catch-all for latest stable version
- `torbrowser-*-stable/version` : platform-specific stable versions
- `torbrowser-*-alpha/version` : platform-specific alpha versions
- `tor-stable`,`tor-alpha` : set by tor devs, do not touch
- [x] Push to origin as new branch, open 'Draft :' MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [ ] Publish after CI passes and builds are published
### blog: https://gitlab.torproject.org/tpo/web/blog.git
- [x] Duplicate previous Stable or Alpha release blog post as appropriate to new directory under `content/blog/new-release-tor-browser-$(TOR_BROWSER_VERSION)` and update with info on release :
- [ ] Run `tools/signing/create-blog-post` which should create the new blog post from a template (edit set-config.blog to set you local blog directory)
- [ ] Update Tor Browser version numbers
- [ ] Note any ESR rebase
- [ ] Link to any Firefox security updates from ESR upgrade
- [ ] Link to any Android-specific security backports
- [ ] Note any updates to :
- tor
- OpenSSL
- NoScript
- [ ] Convert ChangeLog.txt to markdown format used here by :
- `tor-browser-build/tools/changelog-format-blog-post`
- [x] Push to origin as new branch, open `Draft:` MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and website has been updated
### tor-announce mailing list
<details>
<summary>email template</summary>
Subject:
New Release: Tor Browser $(TOR_BROWSER_VERSION) (Android, Windows, macOS, Linux)
Body:
Hi everyone,
Tor Browser $(TOR_BROWSER_VERSION) has now been published for all platforms. For details please see our blog post:
- $(BLOG_POST_URL)
</details>
- [x] Email tor-announce mailing list: tor-announce@lists.torproject.org
- **(Optional)** Additional information:
- [ ] Link to any known issues
</details>richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41971Update Tails URL in downloads warning2023-10-03T13:28:23ZdonutsUpdate Tails URL in downloads warningtails.boum.org now redirects to tails.net. We should update the URL the downloads warning points at (and any others I don't know about) to the new official address.
Source: https://tails.net/news/new_domain/index.en.htmltails.boum.org now redirects to tails.net. We should update the URL the downloads warning points at (and any others I don't know about) to the new official address.
Source: https://tails.net/news/new_domain/index.en.htmlrichardrichardhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41292btcpayserver-02: /srv 90% full2023-08-08T17:10:46ZKezbtcpayserver-02: /srv 90% fullnagios is warning about disk space usage on btcpayserver-02. the specific issue is that the docker overlay fs is using most of the /srv partition.
```
root@btcpayserver-02:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev ...nagios is warning about disk space usage on btcpayserver-02. the specific issue is that the docker overlay fs is using most of the /srv partition.
```
root@btcpayserver-02:~# df -h
Filesystem Size Used Avail Use% Mounted on
udev 3.9G 0 3.9G 0% /dev
tmpfs 795M 1.2M 794M 1% /run
/dev/sda1 9.8G 3.3G 6.0G 36% /
tmpfs 3.9G 72K 3.9G 1% /dev/shm
tmpfs 5.0M 0 5.0M 0% /run/lock
tmpfs 3.9G 0 3.9G 0% /tmp
/dev/sdc 69G 59G 7.2G 90% /srv
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/735b6383a11fd94e44c441aa5e103191b89c14cc8855e07ffb40da340b80f928/merged
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/7895c81f2090c42896189ea70665c39b743dfc668d9ccb2ffd6e73a4a441520b/merged
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/1b8d41ee0217e2d24e630345d323f71ac0f65441c0517fa02ff08a5f94232a6d/merged
shm 64M 16K 64M 1% /srv/docker/containers/f0734d0f17976e9ede4ad209f085773ee4e80852287ecd96fa501a8e28626178/mounts/shm
shm 64M 0 64M 0% /srv/docker/containers/cef2842f39d4da80150c219cc71430a07137e8719e2b8324c57a8b2e4d857ed2/mounts/shm
shm 64M 0 64M 0% /srv/docker/containers/2bb6c98d05f06ecd3f7e58c1184d2150c4c1de813c89226a955308521c9d3471/mounts/shm
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/d37fe48eace9b88c00924bcc66fa96dcaf650006d5e1c39c759455abc5921c20/merged
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/75438c97c9e77763eb7cd84a5177041b696845cc1d3807dbeddcead826d37f32/merged
shm 64M 0 64M 0% /srv/docker/containers/c07628f5a627e0d401230e4f9fe8b0d9eb8c614ef7762acc9d81dbda569537ae/mounts/shm
shm 64M 0 64M 0% /srv/docker/containers/6559636f50460a4d26855eb80b1f2c446fafd2b6a2f9b6d4322418518a242259/mounts/shm
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/86a07343fdf41d9309519fc14b1e5f8eb85fb90d275312d89ff0d27f30dee8df/merged
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/333c5454cb901eb8b25f8f20227edec35188659d9098604dd8c7f31c174e16f0/merged
shm 64M 0 64M 0% /srv/docker/containers/f8d04c7284c79b1bd383d64e6b5d0305d0d6272c2eb14fff11340eb8461003da/mounts/shm
shm 64M 0 64M 0% /srv/docker/containers/2c8e162b5ae001a436afd31294f3c8551fca4413c6b81b50643356620d818569/mounts/shm
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/9e572a7553a25246e3f72f46a6f0f0be327b44e2b146d61705ae9a8b866bb5e0/merged
shm 64M 0 64M 0% /srv/docker/containers/7a924b30814683a78b533f0c83bb5a8b6e46dc333bf9cacb1cd71cbf68ae37a5/mounts/shm
overlay 69G 59G 7.2G 90% /srv/docker/overlay2/89084fcfe2ba1e042e250175f6dcc29147e60617b477a60f85a63f862b08b528/merged
shm 64M 0 64M 0% /srv/docker/containers/8de0025ef6ff1d53f7976bc8f7d93856ad87dbe764beac6f6a8824d35cf041cc/mounts/shm
tmpfs 795M 0 795M 0% /run/user/0
```anarcatanarcathttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41289mattlav would like to be notified when donate detects stripe fraud2023-11-02T18:44:48ZKezmattlav would like to be notified when donate detects stripe fraudthe other day @mattlav asked if he could be notified when the donate middleware detects and bans stripe fraud. we could do this really easily by changing the donate fail2ban logging to include a bit more information, and then setting `ac...the other day @mattlav asked if he could be notified when the donate middleware detects and bans stripe fraud. we could do this really easily by changing the donate fail2ban logging to include a bit more information, and then setting `action = %(action_mwl)s` in the fail2ban config. the biggest blocker there is that crm-ext-01 would need an SMTP client setup somewhere and we'd have to plan for crm-ext to send mail.anarcatanarcathttps://gitlab.torproject.org/tpo/core/arti/-/issues/998tor-proto: We sometimes send 2 END cells for the same circuit2023-10-18T11:00:25Zgabi-250tor-proto: We sometimes send 2 END cells for the same circuittor_proto::circuit::test::accept_stream_after_reject sometimes fails with (it is quite hard to repro):
```
thread 'tokio-runtime-worker' panicked at 'Hang on! We're sending an END on a stream where we already sent an END‽', crates/tor-p...tor_proto::circuit::test::accept_stream_after_reject sometimes fails with (it is quite hard to repro):
```
thread 'tokio-runtime-worker' panicked at 'Hang on! We're sending an END on a stream where we already sent an END‽', crates/tor-proto/src/circuit/streammap.rs:
246:17
thread 'circuit::test::accept_stream_after_reject' panicked at 'called `Option::unwrap()` on a `None` value', crates/tor-proto/src/circuit.rs:1992:60
```
This test uses `IncomingStream::reject(end)`, which sends a `ClosePendingStream` control message to the reactor telling it to send an END message on a stream. The reactor also has a mechanism where it implicitly sends an END cell when the stream is dropped. If the stream is dropped, you can no longer call `reject()` on it, so the reactor will only send one END cell. However, if you call `reject()` and later drop the stream, and the reactor doesn't process the `ClosePendingStream` message in time, it is possible for the reactor to get confused and send 2 END cells for the same stream:
* `stream.reject(end);` is called
* `stream` is dropped
* the reactor notices `stream` was dropped (we have a `StreamEnt::Open { rx, ... }` and we recv'd `None` from `rx`). The reactor sends an END cell to the other party
* in the next `run_once()` execution, the reactor notices the `ClosePendingStream` message, and panics because it realizes it has already sent an END cell for the circuit!gabi-250gabi-250https://gitlab.torproject.org/tpo/core/arti/-/issues/996Follow-up from "tor-proto: Make start_conversation_last_hop() use a given hop...2023-08-30T18:58:35Zgabi-250Follow-up from "tor-proto: Make start_conversation_last_hop() use a given hop, not the last."The following discussion from !1469 should be addressed:
- [ ] @nickm started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1469#note_2928796):
> Now that this is public, let's think a little more abo...The following discussion from !1469 should be addressed:
- [ ] @nickm started a [discussion](https://gitlab.torproject.org/tpo/core/arti/-/merge_requests/1469#note_2928796):
> Now that this is public, let's think a little more about the behavior we want from this type.
>
> We're using 0-indexing here, including in our `Display` implementation, but that gets a bit confusing for users when we call the third hop of a circuit "Hop 2". In various places we try to display hops as 1-indexed values, such `"Extending circuit to hop {} with {:?}"`. But in other places we don't.
>
> I suggest that we add a ticket to rework `impl Display` from this type. I'm happy to do so if you think it's a good idea?gabi-250gabi-250