The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2023-09-22T08:16:21Zhttps://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/issues/1Review details endpoint2023-09-22T08:16:21ZHiroReview details endpoint@mattrighetti asked me to give a second review of the details endpoint and on closer look I found a few issues:
on https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/blame/dev/src/metrics/details.rs#L102
I woul...@mattrighetti asked me to give a second review of the details endpoint and on closer look I found a few issues:
on https://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/blame/dev/src/metrics/details.rs#L102
I would remove all the joins. I think maybe to provide the network weights we could either query the latest network_status_entry_weights at the time of the server status (https://gitlab.torproject.org/tpo/network-health/metrics/metrics-sql-tables/-/blob/main/network_status_tables.sql?ref_type=heads#L70) or make a query to VM proxied, up to the server status time (we have both the weight and the fraction calculated from consensuses https://gitlab.torproject.org/tpo/network-health/metrics/descriptorParser/-/blob/main/METRICS.md?ref_type=heads#from-consensuses).
For both relay and bridge what do we need from the server descriptor? I could add that directly into the status so we don't have to make a join query.Mattia RighettiMattia Righettihttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42121Remove "Open in app" feature2023-10-04T18:45:01ZclairehurstRemove "Open in app" feature<!--
* Use this issue template for reporting a new bug.
-->
### Summary
**Summarize the bug encountered concisely.**
There is a feature to open the current site in the relevant already installed app (i.e. youtube.com -> youtube app) whi...<!--
* Use this issue template for reporting a new bug.
-->
### Summary
**Summarize the bug encountered concisely.**
There is a feature to open the current site in the relevant already installed app (i.e. youtube.com -> youtube app) which is a likability issue, potentially de-anonymizing the user by associating the tor session with the new clear net one. This feature should be removed.
### Steps to reproduce:
**How one can reproduce the issue - this is very important.**
1. Open a site (e.g. youtube.com)
2. Go to settings
3. Tap on "Open in app"
4. The app should now open with the current site loaded
### What is the current bug behavior?
**What actually happens.**
The site is loaded into the installed app, potentially de-anonymizing the user
### What is the expected behavior?
**What you want to see instead**
The "Open in app" option removed
### Environment
**Which operating system are you using? For example: Debian GNU/Linux 10.1, Windows 10, Ubuntu Xenial, FreeBSD 12.2, etc.**
**Which installation method did you use? Distribution package (apt, pkg, homebrew), from source tarball, from Git, etc.**
macOS Ventura 13.5.2, Pixel 7 pro API 34 emulator, installed from source code via git
### Relevant logs and/or screenshots
![Screenshot_2023-09-21_at_2.47.38_PM](/uploads/35312b271ad52e59a1f20525c558285a/Screenshot_2023-09-21_at_2.47.38_PM.png){width=25%}
![Screenshot_2023-09-21_at_2.47.59_PM](/uploads/dda169366dcc590f0ff6f46caaefbdb5/Screenshot_2023-09-21_at_2.47.59_PM.png){width=25%}![Screenshot_2023-09-21_at_3.24.31_PM](/uploads/04097c0f8e9c0656eb5357f8936366fb/Screenshot_2023-09-21_at_3.24.31_PM.png)clairehurstclairehursthttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40960Prepare Mullvad Brower Alpha 13.0a62023-10-03T15:01:01ZrichardPrepare Mullvad Brower Alpha 13.0a6<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a mullvad-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a mullvad-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building mullvad-browser tags, labels, etc
- **example** : `91.6.0`
- `$(MULLVAD_BROWSER_MAJOR)` : the Mullvad Browser major version
- **example** : `11`
- `$(MULLVAD_BROWSER_MINOR)` : the Mullvad Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(MULLVAD_BROWSER_VERSION)` : the Mullvad Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(MULLVAD_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(MULLVAD_BROWSER_BUILD_N)` : the mullvad-browser build revision for a given Mullvad Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(MULLVAD_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For **example** :
- if we have multiple Mullvad Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(MULLVAD_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(MULLVAD_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `mullvad-browser`, the `$(MULLVAD_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(MULLVAD_BROWSER_VERSION)` : the published Mullvad Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(MB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Mullvad Browser version
- **example** : `mb-12.0.7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` alpha rebase and security backport tasks have been completed
**NOTE** This can/is often done in conjunction with the equivalent Tor Browser release prep issue
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Mullvad Browser Alpha (and Nightly) are on the `main` branch
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(MULLVAD_BROWSER_BUILD_N)`
- [x] `var/torbrowser_incremental_from` : update to previous Desktop version
- **NOTE**: We try to build incrementals for the previous 3 desktop versions except in the case of a watershed update
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make mullvadbrowser-incrementals-*` step will fail
- [x] Update build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `mullvad-browser` tag
- [ ] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-alpha` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [ ] `steps/mullvad-browser/git_hash` : update with `HEAD` commit of project's `mullvad-browser` branch
- [ ] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for uBlock-origin updates here : https://addons.mozilla.org/en-US/firefox/addon/ublock-origin/
- [x] ***(Optional)*** If new version available, update `ublock-origin` section of `input_files` in `projects/browser/config`
- [x] `URL`
- [x] `sha256sum`
- [x] Check for Mullvad Privacy Companion updates here : https://github.com/mullvad/browser-extension/releases
- [ ] ***(Optional)*** If new version available, update `mullvad-extension` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Update `ChangeLog-MB.txt`
- [ ] Ensure ChangeLog-MB.txt is sync'd between alpha and stable branches
- [ ] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [ ] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [ ] Copy the output of the script to the beginning of `ChangeLog-MB.txt` and update its output
- [ ] Version
- [ ] Browser Name
- [ ] Release Date
- [ ] Under `All Platforms` include any version updates for:
- NoScript
- uBlock-origin
- Mullvad Browser Extension
- Firefox
- [x] Open MR with above changes
- [x] Build the MR after initial review on at least two of:
- [x] Tor Project build machine
- [ ] Mullvad build machine
- [x] Local developer machine
- [x] Ensure builders have matching builds
- [x] Merge
- [x] Sign+Tag
- **NOTE** this must be done by one of:
- boklm
- dan
- ma1
- pierov
- richard
- [x] Run: `make mullvadbrowser-signtag-alpha`
- [x] Push tag to `origin`
</details>
<details>
<summary>QA</summary>
### send the build
- [x] Email Mullvad QA: support@mullvad.net, rui@mullvad.net
<details>
<summary>email template</summary>
Subject:
New build: Mullvad Browser $(MULLVAD_BROWSER_VERION) (unsigned)
Body:
unsigned builds: https://tb-build-05.torproject.org/~$(BUILDER)/builds/mullvadbrowser/alpha/unsigned/$(MB_BUILD_TAG)
changelog:
...
</details>
- ***(Optional)*** Add additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
</details>
<details>
<summary>Signing</summary>
### signing
- [x] Assign this issue to the signer, one of:
- boklm
- richard
- [x] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a mullvad notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/mullvad-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : mullvad browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.mullvadbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [x] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component dist.torproject.org`
- [x] Remove old release data from `/srv/dist-master.torproject.org/htdocs/mullvadbrowser`
- [x] Static update components (again) : `static-update-component dist.torproject.org`
</details>
<details>
<summary>Publishing</summary>
### mullvad-browser (github): https://github.com/mullvad/mullvad-browser/
- [x] Assign this issue to someone with mullvad commit access, one of:
- richard
- [x] Push this release's associated `mullvad-browser.git` branch to github
- [x] Push this release's associated tags to github:
- [ ] Firefox ESR tag
- **example** : `FIREFOX_102_12_0esr_BUILD1,`
- [ ] `base-browser` tag
- **example** : `base-browser-102.12.0esr-12.0-1-build1`
- [x] `mullvad-browser` tag
- **example** : `mullvad-browser-102.12.0esr-12.0-1-build1`
- [x] Sign+Tag additionally the `mullvad-browser.git` `firefox` commit used in build:
- **Tag**: `$(MULLVAD_BROWSER_VERSION)`
- **example** : `12.5a7`
- **Message**: `$(ESR_VERSION)esr-based $(MULLVAD_BROWSER_VERSION)`
- **example** : `102.12.0esr-based 12.5a7`
- [x] Push tag to github
### email
- [x] Email Mullvad with release information: support@mullvad.net, rui@mullvad.net
<details>
<summary>email template</summary>
Subject:
New build: Mullvad Browser $(MULLVAD_BROWSER_VERION) (signed)
Body:
signed builds: https://dist.torproject.org/mullvadbrowser/$(MULLVAD_BROWSER_VERSION)
update_response hashes: $(MULLVAD_UPDATE_RESPONSES_HASH)
changelog:
...
</details>
</details>
<details>
<summary>Downstream</summary>
### notify packagers
- [ ] **(Optional, Once Mullvad Updates their Github Releases Page)** Email downstream consumers:
- **NOTE**: This is an optional step and only necessary close a major release/transition from alpha to stable, or if there are major packing changes these developers need to be aware of
<details>
<summary>email template</summary>
Hello!
Mullvad-Browser $(MULLVAD_BROWSER_VERSION) packages are available, so you should all update your respective downstream packages.
Release builds can be found here:
- https://github.com/mullvad/mullvad-browser/releases/tag/$(MULLVAD_BROWSER_VERSION)
</details>
- flathub package maintainer: proletarius101@protonmail.com
- arch package maintainer: bootctl@gmail.com
- nixOS package maintainer: dev@felschr.com
</details>richardrichardhttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40959Prepare Tor Browser Alpha 13.0a62023-10-03T15:01:04ZrichardPrepare Tor Browser Alpha 13.0a6<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** :...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building tor-browser tags, labels, etc
- **example** : `91.6.0`
- `$(TOR_BROWSER_MAJOR)` : the Tor Browser major version
- **example** : `11`
- `$(TOR_BROWSER_MINOR)` : the Tor Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(TOR_BROWSER_VERSION)` : the Tor Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(TOR_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(TOR_BROWSER_BUILD_N)` : the tor-browser build revision for a given Tor Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(TOR_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For example :
- if we have multiple Tor Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(TOR_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(TOR_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `tor-browser`, the `$(TOR_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(TOR_BROWSER_VERSION)` : the published Tor Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(TBB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Tor Browser version
- **example** : `tbb-12.5a7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` stable rebase and security backport tasks have been completed
**NOTE** This can/is often done in conjunction with the equivalent Mullvad Browser release prep issue
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Tor Browser Alpha (and Nightly) are on the `main` branch
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [x] `var/torbrowser_build` : update to `$(TOR_BROWSER_BUILD_N)`
- [x] ***(Desktop Only)***`var/torbrowser_incremental_from` : update to previous Desktop version
- **NOTE**: We try to build incrementals for the previous 3 desktop versions except in the case of a watershed update
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make torbrowser-incrementals-*` step will fail
- [x] Update Desktop-specific build configs
- [x] Update `projects/firefox/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-alpha` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [x] `steps/tor-browser/git_hash` : update with `HEAD` commit of project's `tor-browser` branch
- [x] `steps/fenix/git_hash` : update with `HEAD` commit of project's `fenix-torbrowserstringsxml` branch
- [x] Update Android-specific build configs
- [x] Update `projects/geckoview/config`
- [x] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/geckoview_version` : update to latest `$(ESR_VERSION)` if rebased
- [ ] ***(Optional)*** Update `projects/tor-android-service/config`
- [ ] `git_hash` : update with `HEAD` commit of project's `main` branch
- [ ] ***(Optional)*** Update `projects/application-services/config`:
**NOTE** we don't currently have any of our own patches for this project
- [ ] `git_hash` : update to appropriate git commit associated with `$(ESR_VERSION)`
- [x] ***(Optional)*** Update `projects/firefox-android/config`:
- [x] `fenix_version` : update to match alpha `firefox-android` build tag
- [ ] `browser_branch` : update to match alpha `firefox-android` build tag
- [x] Update allowed_addons.json by running (from `tor-browser-build` root):
- `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
- [x] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for OpenSSL updates here : https://www.openssl.org/source/
- [ ] ***(Optional)*** If new 3.0.X version available, update `projects/openssl/config`
- [ ] `version` : update to next 3.0.X version
- [ ] `input_files/sha256sum` : update to sha256 sum of source tarball
- [x] Check for zlib updates here: https://github.com/madler/zlib/releases
- [ ] **(Optional)** If new tag available, update `projects/zlib/config`
- [ ] `version` : update to next release tag
- [x] Check for tor updates here : https://gitlab.torproject.org/tpo/core/tor/-/tags
- [x] ***(Optional)*** Update `projects/tor/config`
- [x] `version` : update to latest `-alpha` tag or release tag if newer (ping dgoulet or ahf if unsure)
- [x] Check for go updates here : https://golang.org/dl
- **NOTE** : Tor Browser Alpha uses the latest Stable major series go version
- [ ] ***(Optional)*** Update `projects/go/config`
- [ ] `version` : update go version
- [ ] `input_files/sha256sum` for `go` : update sha256sum of archive (sha256 sums are displayed on the go download page)
- [x] Check for manual updates by running (from `tor-browser-build` root): `./tools/fetch-manual.py`
- [ ] ***(Optional)*** If new version is available:
- [ ] Upload the downloaded `manual_$PIPELINEID.zip` file to people.tpo
- [ ] Update `projects/manual/config`:
- [ ] Change the `version` to `$PIPELINEID`
- [ ] Update `sha256sum` in the `input_files` section
- [ ] ***(Optional)*** Update the URL if you have uploaded to a different people.tpo home
- [x] Update `ChangeLog-TBB.txt`
- [x] Ensure ChangeLog-TBB.txt is sync'd between alpha and stable branches
- [ ] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [ ] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [ ] Copy the output of the script to the beginning of `ChangeLog-TBB.txt` and update its output
- [ ] Version
- [ ] Browser Name
- [ ] Release Date
- [ ] Under `All Platforms` include any version updates for:
- NoScript
- tor
- OpenSSL
- lyrebird
- Snowflake
- [ ] Under `Windows + macOS + Linux` include any version updates for:
- Firefox
- [ ] Under `Android` include any version updates for:
- Geckoview
- [ ] Under `Windows + Android` include any version updates for:
- zlib
- [ ] Under `Build System/All Platforms` include any version updates for:
- Go
- [x] Open MR with above changes
- [x] Build the MR after initial review on at least two of:
- [x] Tor Project build machine
- [ ] Mullvad build machine
- [x] Local developer machine
- [x] Ensure builders have matching builds
- [x] Merge
- [x] Sign_Tag
- **NOTE** this must be done by one of:
- boklm
- dan
- ma1
- pierov
- richard
- [x] Run: `make torbrowser-signtag-alpha`
- [x] Push tag to `origin`
</details>
<details>
<summary>Communications</summary>
### notify stakeholders
- [x] Email tor-qa mailing list: tor-qa@lists.torproject.org
<details>
<summary>email template</summary>
Subject:
Tor Browser $(TOR_BROWSER_VERION) (Android, Windows, macOS, Linux)
Body:
Hello All,
Unsigned Tor Browser $(TOR_BROWSER_VERSION) alpha candidate builds are now available for testing:
- https://tb-build-05.torproject.org/~$(BUILDER)/builds/alpha/unsigned/$(TOR_BROWSER_VERSION)/
The full changelog can be found here:
- https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/raw/$(TBB_BUILD_TAG)/projects/browser/Bundle-Data/Docs-TBB/ChangeLog.txt
</details>
- ***(Optional)*** Additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
- [x] ***(Optional, only around build/packaging changes)*** Email packagers:
- Recipients:
- Tails dev mailing list: tails-dev@boum.org
- Guardian Project: nathan@guardianproject.info
- torbrowser-launcher: micah@micahflee.com
- FreeBSD port: freebsd@sysctl.cz <!-- Gitlab user maxfx -->
- OpenBSD port: caspar@schutijser.com <!-- Gitlab user cschutijser -->
- [ ] Note any changes which may affect packaging/downstream integration
- [ ] Email external partners:
- ***(Optional, after ESR migration)*** Cloudflare: ask-research@cloudflare.com
- **NOTE** : We need to provide them with updated user agent string so they can update their internal machinery to prevent Tor Browser users from getting so many CAPTCHAs
</details>
<details>
<summary>Signing</summary>
### signing
- **NOTE** : In practice, it's most efficient to have the blog post and website updates ready to merge, since signing doesn't take very long
- [x] Assign this issue to the signer, one of:
- boklm
- richard
- [ ] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build` is on the right commit: `git tag -v tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N) && git checkout tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N)`
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a tor notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/tor-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : tor browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.torbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [ ] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Enable update responses : `sudo -u tb-release ./deploy_update_responses-alpha.sh`
- [x] Remove old release data from following places:
- **NOTE** : Skip this step if we need to hold on to older versions for some reason (for example, this is an Andoid or Desktop-only release, or if we need to hold back installers in favor of build-to-build updates if there are signing issues, etc)
- [x] `/srv/cdn-master.torproject.org/htdocs/aus1/torbrowser`
- [x] `/srv/dist-master.torproject.org/htdocs/torbrowser`
- [x] Static update components (again) : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Publish APKs to Google Play:
- Log into https://play.google.com/apps/publish
- Select `Tor Browser (Alpha)` app
- Navigate to `Release > Production` and click `Create new release` button:
- Upload the `*.multi.apk` APKs
- Update Release Name to Tor Browser version number
- Update Release Notes
- Next to 'Release notes', click `Copy from a previous release`
- Edit blog post url to point to most recent blog post
- Save, review, and configure rollout percentage
- [ ] 25% rollout when publishing a scheduled update
- [x] 100% rollout when publishing a security-driven release
- [ ] Update rollout percentage to 100% after confirmed no major issues
</details>
<details>
<summary>Signature verification</summary>
<details>
<summary>Check whether the .exe files got properly signed and timestamped</summary>
```bash
# Point OSSLSIGNCODE to your osslsigncode binary
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
OSSLSIGNCODE=/path/to/osslsigncode
../../../tools/authenticode_check.sh
popd
```
</details>
<details>
<summary>Check whether the MAR files got properly signed</summary>
```bash
# Point NSSDB to your nssdb containing the mar signing certificate
# Point SIGNMAR to your signmar binary
# Point LD_LIBRARY_PATH to your mar-tools directory
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
NSSDB=/path/to/nssdb
SIGNMAR=/path/to/mar-tools/signmar
LD_LIBRARY_PATH=/path/to/mar-tools/
../../../tools/marsigning_check.sh
popd
```
</details>
</details>
<details>
<summary>Publishing</summary>
### website: https://gitlab.torproject.org/tpo/web/tpo.git
- [x] `databags/versions.ini` : Update the downloads versions
- `torbrowser-stable/version` : sort of a catch-all for latest stable version
- `torbrowser-alpha/version` : sort of a catch-all for latest stable version
- `torbrowser-*-stable/version` : platform-specific stable versions
- `torbrowser-*-alpha/version` : platform-specific alpha versions
- `tor-stable`,`tor-alpha` : set by tor devs, do not touch
- [x] Push to origin as new branch, open 'Draft :' MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and builds are published
### blog: https://gitlab.torproject.org/tpo/web/blog.git
- [x] Duplicate previous Stable or Alpha release blog post as appropriate to new directory under `content/blog/new-release-tor-browser-$(TOR_BROWSER_VERSION)` and update with info on release :
- [ ] Run `tools/signing/create-blog-post` which should create the new blog post from a template (edit set-config.blog to set you local blog directory)
- [ ] Update Tor Browser version numbers
- [ ] Note any ESR rebase
- [ ] Link to any Firefox security updates from ESR upgrade
- [ ] Link to any Android-specific security backports
- [ ] Note any updates to :
- tor
- OpenSSL
- NoScript
- [ ] Convert ChangeLog-TBB.txt to markdown format used here by :
- `tor-browser-build/tools/changelog-format-blog-post`
- [x] Push to origin as new branch, open `Draft:` MR
- [x] Remove `Draft:` from MR once signed-packages are uploaded
- [x] Merge
- [x] Publish after CI passes and website has been updated
### tor-announce mailing list
- [x] Email tor-announce mailing list: tor-announce@lists.torproject.org
<details>
<summary>email template</summary>
Subject:
New Release: Tor Browser $(TOR_BROWSER_VERSION) (Android, Windows, macOS, Linux)
Body:
Hi everyone,
Tor Browser $(TOR_BROWSER_VERSION) has now been published for all platforms. For details please see our blog post:
- $(BLOG_POST_URL)
</details>
- **(Optional)** Additional information:
- [ ] Link to any known issues
</details>richardrichardhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41330Create a `lox` user on rdsys-frontend-012023-10-23T18:30:48ZCecylia BocovichCreate a `lox` user on rdsys-frontend-01On the rdsys-frontend-01 machine, we're going with the plan to create a user per service and setup systemd for that user (see https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/167#note_2943424)). We're planning to deploy t...On the rdsys-frontend-01 machine, we're going with the plan to create a user per service and setup systemd for that user (see https://gitlab.torproject.org/tpo/anti-censorship/rdsys/-/issues/167#note_2943424)). We're planning to deploy the lox distributor and would like a user for that service.
cc @meskio @onyinyanganarcatanarcathttps://gitlab.torproject.org/tpo/tpa/prometheus-alerts/-/issues/13Fix Tor DNS EL alert entry2023-09-21T14:18:41ZGeorg KoppenFix Tor DNS EL alert entryAfter https://gitlab.torproject.org/tpo/network-health/metrics/monitoring-and-alerting/-/merge_requests/19 got merged we need to update the respective alert entry.After https://gitlab.torproject.org/tpo/network-health/metrics/monitoring-and-alerting/-/merge_requests/19 got merged we need to update the respective alert entry.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41328build container (AKA "docker") images from scratch inside GitLab CI2023-10-06T17:37:19Zanarcatbuild container (AKA "docker") images from scratch inside GitLab CIso we've had a few issues tracking this in the past, but none directly saying "i want to build containers here please".
we've had one issue to enable the container registry (https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/89) and ...so we've had a few issues tracking this in the past, but none directly saying "i want to build containers here please".
we've had one issue to enable the container registry (https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/89) and one asking for user namespaces (https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/90), but both of those were either too broad or off topic, or are now irrelevant as we're running containers from podman now (https://gitlab.torproject.org/tpo/tpa/team/-/issues/41296 and https://gitlab.torproject.org/tpo/tpa/team/-/issues/41327).
so this issue aims at solving the "let's build a container inside GitLab CI" problem. TPA's current documentation on the matter shows how to do this with kaniko, but as @micah explained elsewhere (https://gitlab.torproject.org/tpo/tpa/container-images/-/merge_requests/1#note_2930961):
> However, to use Kaniko, we'd have to use an upstream container (`gcr.io/kaniko-project/executor:v1.9.0-debug`), which defeats the purpose of building our own containers.
so let's see if we can bootstrap some container trust chain here. this should probably be done inside the https://gitlab.torproject.org/tpo/tpa/container-images/ project, but that's not mandatory.
@micah i hope you don't mind me creating an actual issue for this, i feel it's better than referencing a MR...micahmicah@torproject.orgmicahmicah@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41327test and possibly replace docker with podman in GitLab runners2023-09-25T15:00:57Zanarcattest and possibly replace docker with podman in GitLab runnersGitLab *finally* managed to port the GitLab runner infrastructure to be useable with Podman:
- https://docs.gitlab.com/runner/executors/docker.html#use-podman-to-run-docker-commands
- https://about.gitlab.com/releases/2022/08/22/gitlab-...GitLab *finally* managed to port the GitLab runner infrastructure to be useable with Podman:
- https://docs.gitlab.com/runner/executors/docker.html#use-podman-to-run-docker-commands
- https://about.gitlab.com/releases/2022/08/22/gitlab-15-3-released/#gitlab-runner-153
- https://gitlab.com/gitlab-org/gitlab-runner/-/issues/27119
this could be tremendously useful for us, in many ways:
1. podman makes it much easier to run "rootless" containers, which could significantly improve the security of our runners
2. that, in turn, could make it easier to build container images in runners (see gitlab#123, gitlab#90, gitlab#89 for background on that work, and https://docs.gitlab.com/runner/executors/docker.html#using-podman-to-build-container-images-from-a-dockerfile for the upstream docs)
3. podman doesn't require a daemon, so runner jobs would could directly under systemd which, in turn, might make gitlab-runner upgrades less disruptive
4. podman is simpler than docker and therefore easier to package in Debian, which means the package may be more up to date (for example, upstream docker is at 22.06-beta0, but unstable has 20.10.17, and stable 20.10.5, while podman is at 4.2.0, which is already packaged in experimental, unstable has 3.4.7 and stable 3.0.1)
Unfortunately, GitLab and Podman has fixed their things in version 15.3 (which we run) and 4.2.0 (which we don't), respectively. So we're not quite ready to run this from the Debian side of things. First the 4.2.0 podman release would need to get into unstable, and there testing. *Then* we could see if we can either get a backport running, or setup a bookworm runner, which therefore might make this part of the %"Debian 12 bookworm upgrade" milestone.
See this page for progress on the podman packaging: https://tracker.debian.org/pkg/libpodDebian 12 bookworm upgradeJérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41325Rename chi-node-14 machine2023-09-21T18:38:02ZJérôme Charaouilavamind@torproject.orgRename chi-node-14 machineWhen we moved out of Cymru (tpo/tpa/team#40929), we had physical machine chi-node-14 shipped to Quintex in Dallas and plugged it directly in the new network, without reinstalling the machine.
This ticket tracks the work to rename it to ...When we moved out of Cymru (tpo/tpa/team#40929), we had physical machine chi-node-14 shipped to Quintex in Dallas and plugged it directly in the new network, without reinstalling the machine.
This ticket tracks the work to rename it to ci-runner-x86-03 to get rid of the legacy name and reflect its true usage.Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/network-health/metrics/tagtor/-/issues/24Allow grouping untagged relays2024-03-13T07:38:24ZGeorg KoppenAllow grouping untagged relaysIn #17 we added search functionality to find tagged relays (with arbitrary tags), We should have an option as well to list relays not tagged by anybody.In #17 we added search functionality to find tagged relays (with arbitrary tags), We should have an option as well to list relays not tagged by anybody.TagTor is completed for its scope in Sponsor 112HiroHirohttps://gitlab.torproject.org/tpo/web/donate/-/issues/7Paypal "return to merchant" sends users to donate-api.tpo instead of donate.tpo2023-09-25T03:22:04Zal smithPaypal "return to merchant" sends users to donate-api.tpo instead of donate.tpoI tried to reproduce a donor's issue and found a different problem.
Using Tor Browser, I go to donate.torproject.org > enter a donation > select PayPal
From there, I go through the process to log in to PayPal (1 million captchas)
Once...I tried to reproduce a donor's issue and found a different problem.
Using Tor Browser, I go to donate.torproject.org > enter a donation > select PayPal
From there, I go through the process to log in to PayPal (1 million captchas)
Once I am in, I can get to the point where I select my credit card, and then an error message appears: "Something went wrong. Return to merchant website?" (Edited to add exact error message)
![Screen_Shot_2022-07-19_at_2.14.28_PM](/uploads/5c8602b8f7366f8732c765e6ad25cccc/Screen_Shot_2022-07-19_at_2.14.28_PM.png)
The link leads me to https://donate-api.torproject.org/?token=EC-71D4264140005233T instead of donate.torproject.org. Not only is donate-api the incorrect subdomain, it's also outdated, see screenshot:
![Screen_Shot_2022-07-19_at_1.51.02_PM](/uploads/19a32aa09ca149c7f2fef2018526c380/Screen_Shot_2022-07-19_at_1.51.02_PM.png)
@kez I recognize fixing donating over Tor is a larger project. But can you fix the erroneous link to our website?2023-09-16https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42114Disable Allow sharing current tab URL from Android's Recents screen in privat...2023-10-03T10:54:03ZrichardDisable Allow sharing current tab URL from Android's Recents screen in private browsing modeSeems to be a pixel-only Android feature. See tor-browser#42006Seems to be a pixel-only Android feature. See tor-browser#42006Dan BallardDan Ballardhttps://gitlab.torproject.org/tpo/applications/mullvad-browser/-/issues/238Rebase Mullvad Browser to 115.3.0esr2023-10-04T17:45:21ZrichardRebase Mullvad Browser to 115.3.0esr**NOTE:** All examples in this template reference the rebase from 102.7.0esr to 102.8.0esr
<details>
<summary>Explanation of Variables</summary>
- `$(ESR_VERSION)`: the Mozilla defined ESR version, used in various places for building...**NOTE:** All examples in this template reference the rebase from 102.7.0esr to 102.8.0esr
<details>
<summary>Explanation of Variables</summary>
- `$(ESR_VERSION)`: the Mozilla defined ESR version, used in various places for building mullvad-browser tags, labels, etc
- **Example**: `102.8.0`
- `$(ESR_TAG)`: the Mozilla defined hg (Mercurial) tag associated with `$(ESR_VERSION)`
- **Example**: `FIREFOX_102_8_0esr_RELEASE`
- `$(BROWSER_MAJOR)`: the browser major version
- **Example**: `12`
- `$(BROWSER_MINOR)`: the browser minor version
- **Example**: either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(BASE_BROWSER_BRANCH)`: the full name of the current `base-browser` branch
- **Example**: `base-browser-102.8.0esr-12.5-1`
- `$(BASE_BROWSER_BRANCH_PREV)`: the full name of the previous `base-browser` branch
- **Example**: `base-browser-102.7.0esr-12.5-1`
- `$(BASE_BROWSER_BRANCH_TAG)`: the `base-browser` build tag used as base commit for `mullvad-browser`
- **Example**: `base-browser-102.8.0esr-12.5-1-build1`
- `$(BASE_BROWSER_BRANCH_PREV_TAG)`: the `base-browser` build tag used as base commit for the previous `mullvad-browser`
- **Example**: `base-browser-102.7.0esr-12.5-1-build1`
- `$(MULLVAD_BROWSER_BRANCH)`: the full name of the current `mullvad-browser` branch
- **Example**: `mullvad-browser-102.8.0esr-12.5-1`
- `$(MULLVAD_BROWSER_BRANCH_PREV)`: the full name of the previous `mullvad-browser` branch
- **Example**: `mullvad-browser-102.7.0esr-12.5-1`
</details>
**NOTE:** It is assumed that we've already rebased and tagged `base-browser` alpha and that we've already rebased `mullvad-browser` stable
### **Bookkeeping**
- [x] Link this issue to the appropriate [Release Prep](https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/?sort=updated_desc&state=opened&label_name%5B%5D=Release%20Prep) issue.
### Update Branch Protection Rules
- [x] In [Repository Settings](https://gitlab.torproject.org/tpo/applications/mullvad-browser/-/settings/repository):
- [x] Remove previous alpha `mullvad-browser` branch protection rules (this will prevent pushing new changes to the branches being rebased)
- [x] Create new `mullvad-browser` branch protection rule:
- **Branch**: `mullvad-browser-$(ESR_VERSION)esr-$(BROWSER_MAJOR).$(BROWSER_MINOR)-1*`
- **Example**: `mullvad-browser-102.8.0esr-12.5-1*`
- **Allowed to merge**: `Maintainers`
- **Allowed to push and merge**: `Maintainers`
- **Allowed to force push**: `false`
### **Create and Push New Branch**
- [x] Create new alpha `mullvad-browser` branch from this ESR's alpha `base-browser` tag
- Branch name in the form: `mullvad-browser-$(ESR_VERSION)esr-$(BROWSER_MAJOR).$(BROWSER_MINOR)-1`
- **Example**: `git branch mullvad-browser-102.8.0esr-12.5-1 base-browser-102.8.0esr-12.5-1-build1`
- [x] Push new `mullvad-browser` branch to `upstream`
- [x] Push `base-browser` tag to `upstream`
### **Rebase tor-browser**
- [x] Checkout a new local branch for the `mullvad-browser` rebase
- **Example**: `git branch mullvad-browser-rebase upstream/mullvad-browser-102.8.0esr-12.5-1`
- [x] `mullvad-browser` rebase
- [x] Cherry-pick the previous `mullvad-browser` branch's commit range up to the last `mullvad-browser` `build1` tag
- **Example**: `git cherry-pick base-browser-102.7.0esr-12.5-1-build1..mullvad-browser-102.7.0esr-12.5-1-build1`
- [x] Rebase and autosquash these newly cherry-picked commits
- **Example**: `git rebase --autosquash --interactive upstream/mullvad-browser-102.8.0esr-12.5-1`
- [ ] Cherry-pick remainder of patches after the last `mullvad-browser` `buildN` tag
- **Example**: `git cherry-pick mullvad-browser-102.7.0esr-12.5-1-build1..upstream/mulvad-browser-102.7.0esr-12.5-1`
- [ ] Rebase and autosquash again, this time replacing all `fixup` and `squash` commands with `pick`. The goal here is to have all of the `fixup` and `squash` commits beside the commit which they modify, but kept un-squashed for easy debugging/bisecting.
- **Example**: `git rebase --autosquash --interactive upstream/mullvad-browser-102.8.0esr-12.5-1`
- [x] Compare patch sets to ensure nothing *weird* happened during conflict resolution:
- [x] diff of diffs:
- Do the diff between `current_patchset.diff` and `rebased_patchset.diff` with your preferred difftool and look at differences on lines that starts with + or -
- `git diff $(BASE_BROWSER_BRANCH_PREV_TAG)..$(MULLVAD_BROWSER_BRANCH_PREV) > current_patchset.diff`
- `git diff $(BASE_BROWSER_BRANCH_TAG)..HEAD > rebased_patchset.diff`
- diff `current_patchset.diff` and `rebased_patchset.diff`
- If everything went correctly, the only lines which should differ should be the lines starting with `index abc123...def456` (unless the previous `base-browser` branch includes changes not included in the previous `mullvad-browser` branch)
- [x] rangediff: `git range-diff $(BASE_BROWSER_BRANCH_PREV_TAG)..$(MULLVAD_BROWSER_BRANCH_PREV) $(BASE_BROWSER_BRANCH_TAG)..HEAD`
- **Example**: `git range-diff base-browser-102.7.0esr-12.5-1-build1..upstream/mullvad-browser-102.7.0esr-12.5-1 base-browser-102.8.0esr-12.5-1-build1..HEAD`
- [x] Open MR for the `mullvad-browser` rebase
- [x] Merge
### **Sign and Tag**
- [x] Sign/Tag `HEAD` of the merged `mullvad-browser` branch:
- **Tag**: `mullvad-browser-$(ESR_VERSION)esr-$(BROWSER_MAJOR).$(BROWSER_MINOR)-1-build1`
- **Message**: `Tagging build1 for $(ESR_VERSION)esr-based stable`
- [x] Push tag to `upstream`richardrichardhttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41324Retire ci-runner-x86-012023-09-19T19:49:35ZJérôme Charaouilavamind@torproject.orgRetire ci-runner-x86-01Superseded by new podman-based runner `ci-runner-x86-02`.
1. ~~[ ] announcement~~ (N/A)
2. [x] nagios
3. [x] retire the host in fabric
4. [x] remove from LDAP with `ldapvi`
5. [x] power-grep
6. [x] remove from tor-passwords
7. ~~...Superseded by new podman-based runner `ci-runner-x86-02`.
1. ~~[ ] announcement~~ (N/A)
2. [x] nagios
3. [x] retire the host in fabric
4. [x] remove from LDAP with `ldapvi`
5. [x] power-grep
6. [x] remove from tor-passwords
7. ~~[ ] remove from DNSwl~~ (N/A)
8. [x] remove from docs
9. ~~[ ] remove from racks~~ (N/A)
10. [x] remove from reverse DNSJérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/network-health/team/-/issues/329Archive doctor-modules repository2023-09-21T07:56:45ZGeorg KoppenArchive doctor-modules repositoryThe private doctor-modules repository contains two scripts for v2 onion attack detection which are not applicable anymore in the v3 world and one which is concerned with finding sybils per nickname. We can move that one into our regular ...The private doctor-modules repository contains two scripts for v2 onion attack detection which are not applicable anymore in the v3 world and one which is concerned with finding sybils per nickname. We can move that one into our regular `doctor` repository given that we have a `sybil_checker.py` script already there (which sends notifications to a public mailing list anyway).
We should make that private repo public then and archive it.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/network-health/metrics/networkstatusapi/-/issues/7Make the number of bridges and relays in the config the same data type2023-10-16T07:36:34ZHiroMake the number of bridges and relays in the config the same data typeI have noticed that the number of bridges and relays in the config.json file is provided twice. Once as string and once as integers. Can we make that just once either as strings or integers?I have noticed that the number of bridges and relays in the config.json file is provided twice. Once as string and once as integers. Can we make that just once either as strings or integers?Mattia RighettiMattia Righettihttps://gitlab.torproject.org/tpo/network-health/metrics/exit-scanner/-/issues/40010TorDNSEL is busted since switch to exitmap's 3802a3651cb3a9d4165c0f14d257f47c...2024-02-08T14:16:58ZGeorg KoppenTorDNSEL is busted since switch to exitmap's 3802a3651cb3a9d4165c0f14d257f47c23576dd5Both `exit-scanner` and `exitmap` get regularly updated via puppet (their `main` is fetched). However, the latest `exitmap` changes on 09/12 broke our setup for TorDNSEL as the `exitmap` run does not finish anymore:
```
Directory /tmp is...Both `exit-scanner` and `exitmap` get regularly updated via puppet (their `main` is fetched). However, the latest `exitmap` changes on 09/12 broke our setup for TorDNSEL as the `exitmap` run does not finish anymore:
```
Directory /tmp is not owned by the user or hasn't mask 700 or it's a symlink.
Traceback (most recent call last):
File "/srv/tordnsel.torproject.org/exitscanner/exitscan.py", line 151, in <module>
run()
File "/srv/tordnsel.torproject.org/exitscanner/exitscan.py", line 76, in run
exit_map_out = open(EXITS_PATH + "exitmap.out", "r")
FileNotFoundError: [Errno 2] No such file or directory: '/srv/tordnsel.torproject.org/exitmap.out'
hon3/dist-packages/stem/process.py", line 153, in launch_tor
init_line = tor_process.stdout.readline().decode('utf-8', 'replace').strip()
File "/usr/lib/python3/dist-packages/stem/process.py", line 136, in timeout_handler
raise OSError('reached a %i second timeout without success' % timeout)
OSError: reached a 300 second timeout without success
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/srv/tordnsel.torproject.org/exitscanner/exitmap/./bin/exitmap", line 39, in main
exitmap_main()
File "/srv/tordnsel.torproject.org/exitscanner/exitmap/src/exitmap.py", line 247, in main
socks_port, control_port = bootstrap_tor(args)
File "/srv/tordnsel.torproject.org/exitscanner/exitmap/src/exitmap.py", line 93, in bootstrap_tor
sys.exit(1)
SystemExit: 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/srv/tordnsel.torproject.org/exitscanner/exitmap/./bin/exitmap", line 51, in <module>
main()
File "/srv/tordnsel.torproject.org/exitscanner/exitmap/./bin/exitmap", line 47, in main
shutil.rmtree(args.tor_dir)
File "/usr/lib/python3.9/shutil.py", line 709, in rmtree
onerror(os.lstat, path, sys.exc_info())
File "/usr/lib/python3.9/shutil.py", line 707, in rmtree
orig_st = os.lstat(path)
FileNotFoundError: [Errno 2] No such file or directory: '/tmp/exitmap_tor_datadir-tordnsel'
```
This is caused by https://gitlab.torproject.org/tpo/network-health/exitmap/-/issues/47.
/cc @jugaGeorg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42110Add a utility module for shared UI methods needed for several tor browser com...2023-10-03T10:54:05ZhenryAdd a utility module for shared UI methods needed for several tor browser componentsIn https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42091 we want to share the onion domain shortening in several places.
We want to create a shared module for this method, and as a place to share other tor-related ut...In https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42091 we want to share the onion domain shortening in several places.
We want to create a shared module for this method, and as a place to share other tor-related utility methods if we need them in the future.henryhenryhttps://gitlab.torproject.org/tpo/network-health/doctor/-/issues/40033Add dannenberg back to consensus-health checks2023-09-20T09:00:00ZGeorg KoppenAdd dannenberg back to consensus-health checksAndreas asked us on 01/30/2021 to skip any dir-auth checks for dannenberg due to the overload/DDoS situation. This has calmed down and we should re-add the checks for it. While deploying the change we should make sure no personal mails a...Andreas asked us on 01/30/2021 to skip any dir-auth checks for dannenberg due to the overload/DDoS situation. This has calmed down and we should re-add the checks for it. While deploying the change we should make sure no personal mails are sent for that case: the ones delivered to the mailing list are sufficient.Georg KoppenGeorg Koppenhttps://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/issues/40Sitemap Revision2023-10-10T13:16:59ZharletaSitemap RevisionLink to sitemap:
https://drive.google.com/file/d/1EbLU4-NiTWhVIpGVZt4k9es2FB2ZjG3X/view?usp=sharing
Initial Feedback:
# Feedback on the site map
## General notes
### On entry points
I am thinking these are literally "entry points"—...Link to sitemap:
https://drive.google.com/file/d/1EbLU4-NiTWhVIpGVZt4k9es2FB2ZjG3X/view?usp=sharing
Initial Feedback:
# Feedback on the site map
## General notes
### On entry points
I am thinking these are literally "entry points"—that is, places
that we expect people to start looking for documentation when they
aren't sure where to find it?
If so:
* The `arti` crate itself is probably another entry point.
* I'm thinking that we should ensure that these all refer the reader to
the same information overview.
### On flexibility
I'm looking over this map under the assumption that it isn't too hard to
revise it later on in Arti's lifecycle as we add or remove pages, or
decide to split up a
category. Is that right? If so, great. If, on the other hand, this
map and these categories needs to be "set in stone", then we should
probably spend even more time on them.
(ahf comment: if we know now there are pages we don't need, then it
would be smarter to say it now so they don't spend time making them,
but i do think we have a lot of flexibility here)
(nickm: right; I was thinking about the case where we want to change something
down the road.)
### On external resources
Since we aren't planning to encompass all of the Tor Project's
documentation into a single site right now, we've got to refer to
external documents. With that in
mind, it probably makes sense to think about places where we'll
be referring to other documents in our universe of documents?
In particular, there are a couple of places where I think it makes sense
to call out to other teams' documents, including "More about Tor", and
"Tor Specifications".
## Specific suggestions
I think we might need a "getting started" section somewhere that
effectively asks the user what they want to do, and directs them to
the actual resources they need.
I think that someplace we should have instructions on how to configure
arti, and plan to have a more or less complete explanation of all our
options.
The "reference" section doesn't make a lot of sense to me; most of it is
just a list of development plans from the README. I think maybe that
could be a single section under a new object called "arti: development
status"?
IMO "Test coverage reports" could just get removed from our
documentation entirely.
"Safer Build Options" might belong under a "how to compile arti"
section, which might in turn go under "getting started"
Once we have an RPC API working, we'll probably want to split
"integrating/embedding arti" into "non-rust" and "rust" sections; the
story of how to do this will be very different.
"Using Arti as a Binary" might be thought of as a user manual for
arti-the-program.
It doesn't necessarily make a sense to have "rustdoc crate
documentation" under this hieararchy; we don't have control over how
that is built or laid out, or which parts of it are
## Some alternative categories
Here's my own attempt to make categories for this information. I'm
not on insisting on any aspect of this information architecture;
I'm mainly sharing it in hopes that it might inspire something.
{Diziet writes: I skimread this list and the organisational scheme
looks reasonable to me. I think some of this list may be quite
aspirational. On the whole I think I prefer Nick's information
architecture, below, to the proposal in the google doc.}
* Main arti documentation site
* About Arti
* What is Arti?
* Getting Started
* What to read next
* User guide
* Who is this guide for?
* Getting Arti
* Compiling from source
(Integrate "safer build")
* Using Arti: The bare minimum
* Starting Arti (as a proxy)
* Configuring your applications to use Arti
* Configuring arti
* Troubleshooting
* Basic topics
* Censorship circumvention
* Hosting an Onion Service
* What Arti can't do yet
* Running a relay (not implemented), write later
* CLI Reference manual
* Configuration reference manual
* Description of files on disk
* Integrating Arti
* Three options: RPC, in-Rust APIs, or custom wrappers.
- Which is right for you?
* Custom wrappers
- Compiling for Android
- Compiling for iOS
* Arti RPC (not implemented yet)
- ... there will be lots of stuff under this.
* Arti in Rust
* Understanding Arti's architecture (basics)
* Embedding Arti in a Rust program: tutorials
- Topics in Rust and where to learn more about them.
- Your first arti program: using TorClient to connect to a
website!
- Your second arti program: working with configuration!
- Your third arti program: Writing an onion service!
- More worked examples
- Links
* Where to find more information (links to rustdoc.)
* Contributing to Arti
* Getting started
* setting up your development environment
* fetching arti's source code
* running the tests
* submitting a patch
* code of conduct
* How to get in touch
* Project policies
* support policies
* semver conformance
* MSRV
* coding standards
* MR workflow conventions
* ...?
* Development documentation
* Project status
* planned milestones and their status
(if this even belongs here?)
* Arti's architecture overview
* Tutorials and topic-guides
* how to add new configuration options
* how to add a new crate
* how to expose a new API
* working with experimental features
* ensuring backward compatibility
* how we log
* error handling
* testing and mocking
* ... ?
* Developer tooling overview
* Working with coverage
* fuzzing
* benchmarking
* testing
* chutney
* shadow
* Design notes
* External resources
* Rustdoc documentation for top-level general use Arti crates
* Specifically, `arti`, `arti-client`, and `.
* Rustdoc documentation for special-purpose high-level crates
* Tor specifications
* Resources on the Tor websiteoluchinwenyioluchinwenyi2023-09-25