The Tor Project issueshttps://gitlab.torproject.org/groups/tpo/-/issues2024-01-22T17:39:23Zhttps://gitlab.torproject.org/tpo/core/onionmasq/-/issues/74Don't accept connections until the Tor connection completes (stretch goal)2024-01-22T17:39:23ZetaDon't accept connections until the Tor connection completes (stretch goal)Currently, we always accept incoming TCP connections, and we then have to reset them if the outgoing connection fails. This sucks.
Working around this is going to be quite hard, since it'll require some major rearchitecting of the way w...Currently, we always accept incoming TCP connections, and we then have to reset them if the outgoing connection fails. This sucks.
Working around this is going to be quite hard, since it'll require some major rearchitecting of the way we hook around smoltcp. I think it's worth doing given we've gotten most of the other base functionality done though, as it should remove a whole load of jank from the experience.VPN pre-alpha 05etaetahttps://gitlab.torproject.org/tpo/core/arti/-/issues/1061CI, review use of after_script2023-10-14T23:37:19ZIan Jacksoniwj@torproject.orgCI, review use of after_scriptApparently `after_script` is strange. It seems like it probably doesn't do what we want, or, at least, what our use of it would seem to imply. See https://gitlab.torproject.org/tpo/tpa/team/-/issues/41323.
We should review our `.gitla...Apparently `after_script` is strange. It seems like it probably doesn't do what we want, or, at least, what our use of it would seem to imply. See https://gitlab.torproject.org/tpo/tpa/team/-/issues/41323.
We should review our `.gitlab-ci.yml`'s uses of `after_script` in the light of this information, and the gitlab docs (also linked to in that other ticket).trinity-1686atrinity-1686ahttps://gitlab.torproject.org/tpo/applications/tor-browser-build/-/issues/40977Prepare Tor Browser Stable 13.0.12023-10-31T19:26:12ZboklmPrepare Tor Browser Stable 13.0.1<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** :...<details>
<summary>Explanation of variables</summary>
- `$(BUILD_SERVER)` : the server the main builder is using to build a tor-browser release
- `$(BUILDER)` : whomever is building the release on the $(BUILD_SERVER)
- **example** : `pierov`
- `$(STAGING_SERVER)` : the server the signer is using to to run the signing process
- `$(ESR_VERSION)` : the Mozilla defined ESR version, used in various places for building tor-browser tags, labels, etc
- **example** : `91.6.0`
- `$(TOR_BROWSER_MAJOR)` : the Tor Browser major version
- **example** : `11`
- `$(TOR_BROWSER_MINOR)` : the Tor Browser minor version
- **example** : either `0` or `5`; Alpha's is always `(Stable + 5) % 10`
- `$(TOR_BROWSER_VERSION)` : the Tor Browser version in the format
- **example** : `12.5a3`, `12.0.3`
- `$(BUILD_N)` : a project's build revision within a its branch; this is separate from the `$(TOR_BROWSER_BUILD_N)` value; many of the Firefox-related projects have a `$(BUILD_N)` suffix and may differ between projects even when they contribute to the same build.
- **example** : `build1`
- `$(TOR_BROWSER_BUILD_N)` : the tor-browser build revision for a given Tor Browser release; used in tagging git commits
- **example** : `build2`
- **NOTE** : A project's `$(BUILD_N)` and `$(TOR_BROWSER_BUILD_N)` may be the same, but it is possible for them to diverge. For example :
- if we have multiple Tor Browser releases on a given ESR branch the two will become out of sync as the `$(BUILD_N)` value will increase, while the `$(TOR_BROWSER_BUILD_N)` value may stay at `build1` (but the `$(TOR_BROWSER_VERSION)` will increase)
- if we have build failures unrelated to `tor-browser`, the `$(TOR_BROWSER_BUILD_N)` value will increase while the `$(BUILD_N)` will stay the same.
- `$(TOR_BROWSER_VERSION)` : the published Tor Browser version
- **example** : `11.5a6`, `11.0.7`
- `$(TBB_BUILD_TAG)` : the `tor-browser-build` build tag used to build a given Tor Browser version
- **example** : `tbb-12.0.7-build1`
</details>
**NOTE** It is assumed that the `tor-browser` stable rebase and security backport tasks have been completed
<details>
<summary>Building</summary>
### tor-browser-build: https://gitlab.torproject.org/tpo/applications/tor-browser-build.git
Tor Browser Stable lives in the various `maint-$(TOR_BROWSER_MAJOR).$(TOR_BROWSER_MINOR)` (and possibly more specific) branches
- [x] Update `rbm.conf`
- [x] `var/torbrowser_version` : update to next version
- [ ] `var/torbrowser_build` : update to `$(TOR_BROWSER_BUILD_N)`
- [x] ***(Desktop Only)***`var/torbrowser_incremental_from` : update to previous Desktop version
- **IMPORTANT**: Really *actually* make sure this is the previous Desktop version or else the `make torbrowser-incrementals-*` step will fail
- [ ] Update Desktop-specific build configs
- [x] Update `projects/firefox/config`
- [ ] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/firefox_platform_version` : update to latest `$(ESR_VERSION)` if rebased
- [x] Update `projects/translation/config`:
- [x] run `make list_translation_updates-release` to get updated hashes
- [x] `steps/base-browser/git_hash` : update with `HEAD` commit of project's `base-browser` branch
- [ ] `steps/base-browser-fluent/git_hash` : update with `HEAD` commit of project's `basebrowser-newidentityftl` branch
- [x] `steps/tor-browser/git_hash` : update with `HEAD` commit of project's `tor-browser` branch
- [x] `steps/fenix/git_hash` : update with `HEAD` commit of project's `fenix-torbrowserstringsxml` branch
- [ ] Update Android-specific build configs
- [x] Update `projects/geckoview/config`
- [ ] `browser_build` : update to match `tor-browser` tag
- [ ] ***(Optional)*** `var/geckoview_version` : update to latest `$(ESR_VERSION)` if rebased
- [ ] ***(Optional)*** Update `projects/tor-android-service/config`
- [ ] `git_hash` : update with `HEAD` commit of project's `main` branch
- [ ] ***(Optional)*** Update `projects/application-services/config`:
**NOTE** we don't currently have any of our own patches for this project
- [ ] `git_hash` : update to appropriate git commit associated with `$(ESR_VERSION)`
- [ ] ***(Optional)*** Update `projects/android-components/config`:
- [ ] `android_components_build` : update to match stable android-components tag
- [ ] ***(Optional)*** Update `projects/fenix/config`
- [ ] `fenix_build` : update to match fenix tag
- [x] Update allowed_addons.json by running (from `tor-browser-build` root):
- `./tools/fetch_allowed_addons.py > projects/browser/allowed_addons.json`
- [ ] Update common build configs
- [x] Check for NoScript updates here : https://addons.mozilla.org/en-US/firefox/addon/noscript
- [ ] ***(Optional)*** If new version available, update `noscript` section of `input_files` in `projects/browser/config`
- [ ] `URL`
- [ ] `sha256sum`
- [x] Check for OpenSSL updates here : https://www.openssl.org/source/
- [ ] ***(Optional)*** If new 1.X.Y version available, update `projects/openssl/config`
- [ ] `version` : update to next 1.X.Y version
- [ ] `input_files/sha256sum` : update to sha256 sum of source tarball
- [x] Check for zlib updates here: https://github.com/madler/zlib/releases
- [ ] **(Optional)** If new tag available, update `projects/zlib/config`
- [ ] `version` : update to next release tag
- [x] Check for tor updates here : https://gitlab.torproject.org/tpo/core/tor/-/tags
- [ ] ***(Optional)*** Update `projects/tor/config`
- [ ] `version` : update to latest non `-alpha` tag (ping dgoulet or ahf if unsure)
- [x] Check for go updates here : https://golang.org/dl
- **NOTE** : Tor Browser Stable uses the latest of the *previous* Stable major series go version (apart from the transition phase from Tor Browser Alpha to Stable, in which case Tor Browser Stable may use the latest major series go version)
- [x] ***(Optional)*** Update `projects/go/config`
- [x] `version` : update go version
- [x] `input_files/sha256sum` for `go` : update sha256sum of archive (sha256 sums are displayed on the go download page)
- [x] Check for manual updates by running (from `tor-browser-build` root): `./tools/fetch-manual.py`
- [x] ***(Optional)*** If new version is available:
- [x] Upload the downloaded `manual_$PIPELINEID.zip` file to people.tpo
- [x] Update `projects/manual/config`:
- [x] Change the `version` to `$PIPELINEID`
- [x] Update `sha256sum` in the `input_files` section
- [ ] ***(Optional)*** Update the URL if you have uploaded to a different people.tpo home
- [x] Update `ChangeLog.txt`
- [x] Ensure ChangeLog.txt is sync'd between alpha and stable branches
- [x] Check the linked issues: ask people to check if any are missing, remove the not fixed ones
- [x] Run `tools/fetch-changelogs.py $(TOR_BROWSER_VERSION)` or `tools/fetch-changelogs.py '#$(ISSUE_NUMBER)'`
- Make sure you have `requests` installed (e.g., `apt install python3-requests`)
- The first time you run this script you will need to generate an access token; the script will guide you
- [x] Copy the output of the script to the beginning of `ChangeLog.txt` and adjust its output
- **NOTE** : If you used the issue number, you will need to write the Tor Browser version manually
- [x] ***(Optional)*** Under `All Platforms` include any version updates for:
- [ ] Translations
- [ ] OpenSSL
- [ ] NoScript
- [ ] zlib
- [ ] tor daemon
- [x] ***(Optional)*** Under `Windows + macOS + Linux` include updates for:
- [x] Firefox
- [x] ***(Optional)*** Under `Android`, include updates for:
- [x] Geckoview
- [x] ***(Optional)*** Under `Build System/All Platforms` include updates for:
- [x] Go
- [x] Open MR with above changes
- [x] Merge
- [x] Sign/Tag commit: `make torbrowser-signtag-release`
- [x] Push tag to `origin`
- [x] Begin build on `$(BUILD_SERVER)` (fix any issues in subsequent MRs)
- [ ] **TODO** Submit build-tag to Mullvad build infra
- [x] Ensure builders have matching builds
</details>
<details>
<summary>Communications</summary>
### notify stakeholders
<details>
<summary>email template</summary>
Subject:
Tor Browser $(TOR_BROWSER_VERION) (Android, Windows, macOS, Linux)
Body:
Hello All,
Unsigned Tor Browser $(TOR_BROWSER_VERSION) release candidate builds are now available for testing:
- https://tb-build-05.torproject.org/~$(BUILDER)/builds/release/unsigned/$(TOR_BROWSER_VERSION)/
The full changelog can be found here:
- https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/$(TBB_BUILD_TAG)/ChangeLog.txt
</details>
- [ ] Email tor-qa mailing list: tor-qa@lists.torproject.org
- ***(Optional)*** Additional information:
- [ ] Note any new functionality which needs testing
- [ ] Link to any known issues
- [ ] Email packagers:
- Recipients:
- Tails dev mailing list: tails-dev@boum.org
- Guardian Project: nathan@guardianproject.info
- torbrowser-launcher: micah@micahflee.com
- FreeBSD port: freebsd@sysctl.cz <!-- Gitlab user maxfx -->
- OpenBSD port: caspar@schutijser.com <!-- Gitlab user cschutijser -->
- [ ] ***(Optional)*** Note any changes which may affect packaging/downstream integration
</details>
<details>
<summary>Signing</summary>
### signing
- **NOTE** : In practice, it's most efficient to have the blog post and website updates ready to merge, since signing doesn't take very long
- [x] On `$(STAGING_SERVER)`, ensure updated:
- [x] `tor-browser-build` is on the right commit: `git tag -v tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N) && git checkout tbb-$(TOR_BROWSER_VERSION)-$(TOR_BROWSER_BUILD_N)`
- [x] `tor-browser-build/tools/signing/set-config.hosts`
- `ssh_host_builder` : ssh hostname of machine with unsigned builds
- **NOTE** : `tor-browser-build` is expected to be in the `$HOME` directory)
- `ssh_host_linux_signer` : ssh hostname of linux signing machine
- `ssh_host_macos_signer` : ssh hostname of macOS signing machine
- [x] `tor-browser-build/tools/signing/set-config.macos-notarization`
- `macos_notarization_user` : the email login for a tor notariser Apple Developer account
- [x] `set-config.update-responses`
- `update_responses_repository_dir` : directory where you cloned `git@gitlab.torproject.org:tpo/applications/tor-browser-update-responses.git`
- [x] `tor-browser-build/tools/signing/set-config.tbb-version`
- `tbb_version` : tor browser version string, same as `var/torbrowser_version` in `rbm.conf` (examples: `11.5a12`, `11.0.13`)
- `tbb_version_build` : the tor-browser-build build number (if `var/torbrowser_build` in `rbm.conf` is `buildN` then this value is `N`)
- `tbb_version_type` : either `alpha` for alpha releases or `release` for stable releases
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, run the macOS proxy script:
- `cd tor-browser-build/tools/signing/`
- `./macos-signer-proxy`
- [x] On `$(STAGING_SERVER)` in a separate `screen` session, ensure tor daemon is running with SOCKS5 proxy on the default port 9050
- [x] run do-all-signing script:
- `cd tor-browser-build/tools/signing/`
- `./do-all-signing.torbrowser`
- **NOTE**: at this point the signed binaries should have been copied to `staticiforme`
- [ ] Update `staticiforme.torproject.org`:
- From `screen` session on `staticiforme.torproject.org`:
- [x] Static update components : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [x] Enable update responses : `sudo -u tb-release ./deploy_update_responses-release.sh`
- [x] Remove old release data from following places:
- **NOTE** : Skip this step if we need to hold on to older versions for some reason (for example, this is an Andoid or Desktop-only release, or if we need to hold back installers in favor of build-to-build updates if there are signing issues, etc)
- [x] `/srv/cdn-master.torproject.org/htdocs/aus1/torbrowser`
- [x] `/srv/dist-master.torproject.org/htdocs/torbrowser`
- [ ] Static update components (again) : `static-update-component cdn.torproject.org && static-update-component dist.torproject.org`
- [ ] Publish APKs to Google Play:
- Log into https://play.google.com/apps/publish
- Select `Tor Browser` app
- Navigate to `Release > Production` and click `Create new release` button:
- Upload the `*.multi.apk` APKs
- Update Release Name to Tor Browser version number
- Update Release Notes
- Next to 'Release notes', click `Copy from a previous release`
- Edit blog post url to point to most recent blog post
- Save, review, and configure rollout percentage
- [ ] 25% rollout when publishing a scheduled update
- [ ] 100% rollout when publishing a security-driven release
- [ ] Update rollout percentage to 100% after confirmed no major issues
</details>
<details>
<summary>Signature verification</summary>
<details>
<summary>Check whether the .exe files got properly signed and timestamped</summary>
```
# Point OSSLSIGNCODE to your osslsigncode binary
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
OSSLSIGNCODE=/path/to/osslsigncode
../../../tools/authenticode_check.sh
popd
```
</details>
<details>
<summary>Check whether the MAR files got properly signed</summary>
```
# Point NSSDB to your nssdb containing the mar signing certificate
# Point SIGNMAR to your signmar binary
# Point LD_LIBRARY_PATH to your mar-tools directory
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
NSSDB=/path/to/nssdb
SIGNMAR=/path/to/mar-tools/signmar
LD_LIBRARY_PATH=/path/to/mar-tools/
../../../tools/marsigning_check.sh
popd
```
</details>
</details>
<details>
<summary>Publishing</summary>
### website: https://gitlab.torproject.org/tpo/web/tpo.git
- [x] `databags/versions.ini` : Update the downloads versions
- `torbrowser-stable/version` : sort of a catch-all for latest stable version
- `torbrowser-alpha/version` : sort of a catch-all for latest stable version
- `torbrowser-*-stable/version` : platform-specific stable versions
- `torbrowser-*-alpha/version` : platform-specific alpha versions
- `tor-stable`,`tor-alpha` : set by tor devs, do not touch
- [x] Push to origin as new branch, open 'Draft :' MR
- [ ] Remove `Draft:` from MR once signed-packages are uploaded
- [ ] Merge
- [ ] Publish after CI passes and builds are published
### blog: https://gitlab.torproject.org/tpo/web/blog.git
- [x] Duplicate previous Stable or Alpha release blog post as appropriate to new directory under `content/blog/new-release-tor-browser-$(TOR_BROWSER_VERSION)` and update with info on release :
- [ ] Run `tools/signing/create-blog-post` which should create the new blog post from a template (edit set-config.blog to set you local blog directory)
- [ ] Update Tor Browser version numbers
- [ ] Note any ESR rebase
- [ ] Link to any Firefox security updates from ESR upgrade
- [ ] Link to any Android-specific security backports
- [ ] Note any updates to :
- tor
- OpenSSL
- NoScript
- [ ] Convert ChangeLog.txt to markdown format used here by :
- `tor-browser-build/tools/changelog-format-blog-post`
- [x] Push to origin as new branch, open `Draft:` MR
- [ ] Remove `Draft:` from MR once signed-packages are uploaded
- [ ] Merge
- [ ] Publish after CI passes and website has been updated
### tor-announce mailing list
<details>
<summary>email template</summary>
Subject:
New Release: Tor Browser $(TOR_BROWSER_VERSION) (Android, Windows, macOS, Linux)
Body:
Hi everyone,
Tor Browser $(TOR_BROWSER_VERSION) has now been published for all platforms. For details please see our blog post:
- $(BLOG_POST_URL)
</details>
- [ ] Email tor-announce mailing list: tor-announce@lists.torproject.org
- **(Optional)** Additional information:
- [ ] Link to any known issues
</details>richardrichardhttps://gitlab.torproject.org/tpo/network-health/sbws/-/issues/40178Check and document how to send email notifications when debian systemd timers...2024-02-05T18:28:27ZjugaCheck and document how to send email notifications when debian systemd timers failsbws: 1.9.x-finaljugajugahttps://gitlab.torproject.org/tpo/tpa/team/-/issues/41356GitHub mirror for Onionbalance2023-10-18T16:55:55ZSilvio RhattoGitHub mirror for OnionbalanceThe [Onionbalance](https://onionbalance.readthedocs.io) codebase was recently [moved around](tpo/onion-services/onionbalance#10):
* From https://github.com/asn-d6/onionbalance to https://github.com/torproject/onionbalance.
* From https:...The [Onionbalance](https://onionbalance.readthedocs.io) codebase was recently [moved around](tpo/onion-services/onionbalance#10):
* From https://github.com/asn-d6/onionbalance to https://github.com/torproject/onionbalance.
* From https://gitlab.torproject.org/tpo/core/onionbalance to https://gitlab.torproject.org/tpo/onion-services/onionbalance.
Now I'd like to configure automatic pushes from Tor's GitLab to GitHub.
I'm aware of the [recommended procedure](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/gitlab#how-to-mirror-a-git-repository-from-gitlab-to-github) and have done it before, but I don't have admin access in the [GitHub repo](https://github.com/torproject/onionbalance) to proceed (my user on GitHub is [rhatto](https://github.com/rhatto/)).
Not sure if this is the right place to ask, but did so after finding a related issue (tpo/tpa/team#41246).Jérôme Charaouilavamind@torproject.orgJérôme Charaouilavamind@torproject.orghttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42167Make the preference auto-focus more reliable2023-10-11T12:23:15ZhenryMake the preference auto-focus more reliableRight now testing with NVDA, the auto-focus implemented for https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41454 was not always reliable in getting the screen reader to read the Security Level panel.Right now testing with NVDA, the auto-focus implemented for https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/41454 was not always reliable in getting the screen reader to read the Security Level panel.henryhenryhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42166New identity dialog missing accessible name2023-10-11T12:22:45ZhenryNew identity dialog missing accessible nameThe new identity dialog is missing an accessible name, causing it to use the "chrome:" path as the dialog as a fallback.The new identity dialog is missing an accessible name, causing it to use the "chrome:" path as the dialog as a fallback.henryhenryhttps://gitlab.torproject.org/tpo/applications/rbm/-/issues/40062Copy input directories to containers recursively2023-10-13T05:17:30ZPier Angelo VendrameCopy input directories to containers recursivelyEarlier I've run a `firefox-android` build with `RBM_VERBOSE_LOG=1`.
I found that indeed we spend a [very long time copying files](/uploads/974c39037b48eb7f94954e4ee18194c7/android-copies.txt).
They are about 1600 files for slightly les...Earlier I've run a `firefox-android` build with `RBM_VERBOSE_LOG=1`.
I found that indeed we spend a [very long time copying files](/uploads/974c39037b48eb7f94954e4ee18194c7/android-copies.txt).
They are about 1600 files for slightly less than 500MB, so all this time is quite surprising.
My hypothesis is that it takes us this long time because we copy them one by one and we adjust their owner after each copy.
I wonder if this adds a lot of overhead (we need to setup the container and chroot to it to run `chown`).
My proposal is that we first copy directories (such as the various `gradle-dependencies-N`) recursively, and that we apply the owner only at the end of the copy.https://gitlab.torproject.org/tpo/network-health/tor-weather/-/issues/82Fix Tor Weather pipeline (pip install overrides poetry.lock pins)2023-11-23T10:02:49ZGeorg KoppenFix Tor Weather pipeline (pip install overrides poetry.lock pins)Building the Tor Weather wheel is working fine. However, we are failing in the `Verify` stage. While we do pin hashes in `poetry.lock` this gets bypassed by `python3 -m pip install dist/tor_weather-*.whl` which grabs the latest available...Building the Tor Weather wheel is working fine. However, we are failing in the `Verify` stage. While we do pin hashes in `poetry.lock` this gets bypassed by `python3 -m pip install dist/tor_weather-*.whl` which grabs the latest available versions it seems potentially resulting in a version mismatch and broken Tor Weather setups. For instance, right now we pin `flask` to version 2.3.3 but running `python3 -m pip install dist/tor_weather-*.whl` in our pipeline actually results in
```
Collecting flask (from tor-weather==1.1.1)
Obtaining dependency information for flask from https://files.pythonhosted.org/packages/36/42/015c23096649b908c809c69388a805a571a3bea44362fe87e33fc3afa01f/flask-3.0.0-py3-none-any.whl.metadata
Downloading flask-3.0.0-py3-none-any.whl.metadata (3.6 kB)
```Sarthik Guptasarthikg@icloud.comSarthik Guptasarthikg@icloud.comhttps://gitlab.torproject.org/tpo/core/tor/-/issues/40871Tor incorrectly stores stats on incoming PT connections2023-12-10T21:38:18ZAlexander Færøyahf@torproject.orgTor incorrectly stores stats on incoming PT connections@trinity-1686a and @dcf discussed this issue on tor-dev@ in https://lists.torproject.org/pipermail/tor-dev/2023-October/014858.html
It seems like we have a bug after we updated our connectiong tracking code to track incoming connections...@trinity-1686a and @dcf discussed this issue on tor-dev@ in https://lists.torproject.org/pipermail/tor-dev/2023-October/014858.html
It seems like we have a bug after we updated our connectiong tracking code to track incoming connections earlier. We don't handle the transport name parameter of our eager call to `geoip_note_client_seen()`.
@trinity-1686a may potentially have a patch for this. I think it would be good if we could get some testing on this before we merge it.
Would you be up for running your Tor instance with a patch that potentially fixes this issue, @dcf ?Tor: 0.4.8.x-post-stabletrinity-1686atrinity-1686ahttps://gitlab.torproject.org/tpo/core/arti/-/issues/1059futures::channel::oneshot severe bug affecting cleanup2023-10-11T16:02:52ZIan Jacksoniwj@torproject.orgfutures::channel::oneshot severe bug affecting cleanup`futures::channel::oneshot::Receiver` has a broken implementation of `FusedFuture` which makes it not work in `select!`. If the `Sender` is dropped, the `select!` functions as if the `Receiver` is never `Ready` (even though when you pol...`futures::channel::oneshot::Receiver` has a broken implementation of `FusedFuture` which makes it not work in `select!`. If the `Sender` is dropped, the `select!` functions as if the `Receiver` is never `Ready` (even though when you poll it it *is* `Ready` in this case). https://github.com/rust-lang/futures-rs/issues/2455
One big problem with this that on shutdown of part of Arti, it can cause tasks to hang rather than noticing everything has gone away and terminating.
Options:
1. Do something (what?) to try to spot when a `oneshot::Receiver` is used within `select!`, and call `.fuse()` on it (and store that somewhere) to get a properly working impl of `FusedFuture`. I don't think this is very viable unless we have a more concrete idea. (Hrm, can we make `<Receiver as FusedFuture>::is_terminated` a deprecated method and spot its uses by `select!`?)
2. Use Tokio's `oneshot`. Inspection of the implementation (thanks @trinity-1686a) reveals that it doesn't depend on the Tokio executor. Downside: every crate now ends up with a dep on Tokio. This is bad because it would make it easy to use other bits of tokio, despite our intent to be runtime-agnostic. It's also bad for our code size.
3. Use Tokio's `oneshot` but via a re-export it from `tor-async-utils.` That helps prevent the accidental use of Tokio but it doesn't help with code size.
4. Copy Tokio's `oneshot.rs` into `tor-async-utils`.
5. Use `postage::oneshot`. But it has a different (and rather less good) API.
6. Use `postage::oneshot` but wrap it in a newtype. I'm not sure the implementation of the good API in terms of the less good one will be much fun or very principled.
7. Write our own `oneshot`.
8. Use `FusedFuture<oneshot::Receiver<T>>` everywhere and enforce this by (i) making all the constructors like `oneshot::channel` deprecated and (ii) providing wrappers that wrap the `Receiver`. (This is an extra byte (probably word) of state for each `Receiver` and therefore will have a perf impact, so on super hot paths we might want something else. But we don't have many oneshots on super hot paths.)
CC @gabi-250 @nickm since this'll probably end up being a widespread thing.Ian Jacksoniwj@torproject.orgIan Jacksoniwj@torproject.orghttps://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/issues/63[New] examples (a page with list of example projects from the repo)2023-12-20T13:40:34Zharleta[New] examples (a page with list of example projects from the repo)2023-10-14https://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/issues/62[New] overview (Arti three ways)2023-12-20T13:41:51Zharleta[New] overview (Arti three ways)https://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/blob/c77837e821cbfe8a76cf2c24025cff16bf7605c7/docs/integrating-arti/integrating-arti.mdhttps://gitlab.torproject.org/tpo/core/arti-doc-project-2023/-/blob/c77837e821cbfe8a76cf2c24025cff16bf7605c7/docs/integrating-arti/integrating-arti.md2023-10-14https://gitlab.torproject.org/tpo/network-health/metrics/website/-/issues/40102tor-0.4.8 overestimates <OR> bridge users and underestimates PT bridge users2024-01-25T14:33:21ZDavid Fifielddcf@torproject.orgtor-0.4.8 overestimates <OR> bridge users and underestimates PT bridge usersLook at the graphs below.
As bridges have upgraded from tor-0.4.7 to tor-0.4.8,
two things have happened simultaneously:
1. The estimated number of bridge users using the \<OR\> protocol (no pluggable transport) has gone up.
2. The esti...Look at the graphs below.
As bridges have upgraded from tor-0.4.7 to tor-0.4.8,
two things have happened simultaneously:
1. The estimated number of bridge users using the \<OR\> protocol (no pluggable transport) has gone up.
2. The estimated number of bridge users using any pluggable transports has gone down.
It is no coincidence.
A bug was introduced in tor-0.4.8 that apparently causes the \<OR\> counter in
`bridge-ip-transports` to be incremented
every time the counter for any transport is incremented.
Bridges that formerly reported 0% \<OR\> and 100% transport <var>T</var>
are now reporting around 50% \<OR\> and 50% transport <var>T</var>.
Because per-transport directory requests are estimated
[according to the *ratios*](https://metrics.torproject.org/reproducible-metrics.html#bridge-users)
in `bridge-ip-transports`, this makes it appear as if half the bridge's users
are \<OR\> and half are transport <var>T</var>,
which makes the total \<OR\> count go up and the total pluggable transport count go down.
I believe the bug is in tor, not in metrics code,
but I'm opening an issue here because I don't know the cause
of the bug in tor yet, and because metrics may have to be aware
of the erroneous descriptors that have already been published.
@trinity-1686a [suspects](https://lists.torproject.org/pipermail/tor-dev/2023-October/014856.html)
commit https://gitlab.torproject.org/tpo/core/tor/-/commit/3e18507dc75afcf0c6560e966c9f18942406b0c8,
which would make the first affected release
[tor-0.4.8.4](https://forum.torproject.org/t/stable-release-0-4-8-4/8884)
on 2023-08-23.
[![Relay versions](/uploads/e1f1e65b138bcb6d12b026bf216f5015/versions-2023-07-11-2023-10-09.png){width=640px}](https://metrics.torproject.org/versions.html?start=2023-07-11&end=2023-10-09)
[![Bridge users by transport](/uploads/1a82ff4038d032771657f8dd26db0bc3/userstats-bridge-transport-2023-07-11-2023-10-09-__OR_-_OR_.png){width=640px}](https://metrics.torproject.org/userstats-bridge-transport.html?start=2023-07-11&end=2023-10-09&transport=%21%3COR%3E&transport=%3COR%3E)
I noticed this after tor upgrades on the Snowflake bridges.
Here are some excerpts from the thread on tor-dev.
https://lists.torproject.org/pipermail/tor-dev/2023-October/014855.html
> I upgraded one of the two Snowflake bridges from 0.4.7.13 to 0.4.8.6 on
> 2023-09-24. Since then, the number of \<OR\> IP addresses has been roughly
> equal to the number of snowflake IP addresses. The ORPort is still not
> exposed; these are not external vanilla bridge users. Did something
> change between these versions that might cause PT connections to be
> double-counted, once for the transport and once for \<OR\>?
>
> Here are excerpted bridge-extra-info descriptors from before and after
> the version upgrade. Note the `bridge-ip-transports` lines.
>
> ```
> @type bridge-extra-info 1.3
> extra-info crusty5 91DA221A149007D0FD9E5515F5786C3DD07E4BB0
> master-key-ed25519 1y9CAtinlbrhDuYBSOBNiCU9Ck1lcY7LErxnzhtxVks
> published 2023-09-19 10:46:08
> transport snowflake
> bridge-ip-versions v4=13528,v6=1384
> bridge-ip-transports <OR>=8,snowflake=14912
> ```
>
> ```
> @type bridge-extra-info 1.3
> extra-info crusty5 91DA221A149007D0FD9E5515F5786C3DD07E4BB0
> master-key-ed25519 1y9CAtinlbrhDuYBSOBNiCU9Ck1lcY7LErxnzhtxVks
> published 2023-09-29 17:33:20
> transport snowflake
> bridge-ip-versions v4=2880,v6=336
> bridge-ip-transports <OR>=1632,snowflake=1592
> ```
https://lists.torproject.org/pipermail/tor-dev/2023-October/014858.html
> The effect of this apparent bug is that the lower bound of per-country
> per-transport user count intervals goes to zero. See the first attached
> graph. The snowflake-01 bridge upgraded to 0.4.8 on 2023-10-03;
> snowflake-02 upgraded on 2023-09-24. Whereas formerly, the low–high
> intervals were so narrow as to be indistinguishable from a line, now
> they extend all the way down to the <var>x</var>-axis.
>
> ![Top 5 countries with the most Snowflake users by day](/uploads/f9f0ce55de779dc30ddb9cd1b099a80a/snowflake-top-countries-20231010.png){width=640px}
>
> The [formula for the lower bound](https://metrics.torproject.org/reproducible-metrics.html#bridge-users) is:
>
> > low = max(0, country_reqs + transport_reqs − total_reqs)
>
> On a bridge that hides its ORPort and runs just one transport, we have
> transport_reqs ≈ total_reqs, and so the above just becomes
>
> > low = max(0, country_reqs)
>
> But with the apparent metrics bug in 0.4.8.6, total_reqs is twice as
> large as it should be (i.e., 2 × transport_reqs ≈ total_reqs), which
> means the formula becomes
>
> > low = max(0, country_reqs − transport_reqs)
>
> which, since country_reqs < transport_reqs, always becomes zero.
>
> The upper bound of the interval is unaffected; its formula is
>
> > high = min(country_reqs, transport_reqs)
>
> The other side effect is that directory requests that ought to be
> attributed entirely to a pluggable transport are instead being ascribed
> 50/50 to that transport and the plain OR protocol. "We approximate
> [directory requests by transport] by multiplying the total number of
> requests with the fraction of unique IP addresses by transport." See, in
> the second attached graph, how estimated pluggable transport users have
> declined and OR protocol users have increased, in an almost inverse
> relationship, roughly in step with the 0.4.7→0.4.8 upgrade.
>
> https://metrics.torproject.org/userstats-bridge-transport.html?start=2023-07-11&end=2023-10-09&transport=%21%3COR%3E&transport=%3COR%3E
> https://metrics.torproject.org/versions.html?start=2023-07-11&end=2023-10-09HiroHiro2024-01-31https://gitlab.torproject.org/tpo/community/hackweek/-/issues/18Working on the content for the developer portal2023-11-30T16:16:39ZGabagaba@torproject.orgWorking on the content for the developer portal# About the project
* Contact: @gaba
* Chat: #tor-dev on `irc.oftc.net`
* Video room: https://tor.meet.coop/gab-tph-u9q-eo0
* Meet Monday, Tuesday, Wednesday, Thursday from 12UTC to 20UTC
# Participants
- @gaba
- you?
# Summary
...# About the project
* Contact: @gaba
* Chat: #tor-dev on `irc.oftc.net`
* Video room: https://tor.meet.coop/gab-tph-u9q-eo0
* Meet Monday, Tuesday, Wednesday, Thursday from 12UTC to 20UTC
# Participants
- @gaba
- you?
# Summary
The [developer portal](https://gitlab.torproject.org/tpo/web/dev/) has been on the back waiting for some time to get completed. At the beginning of this year Ura.Design worked on a [site/design](https://gitlab.torproject.org/tpo/web/dev/-/issues/6) for it. This is a project to get the content into the site and reviewing the information architecture.
## Project A : Get content into the portal
The content that we are planning to have in this portal is all over the place in repositories and wikis. With this project I will move the content to the dev portal, review the information architecture and organize the work that needs to happen next.
# Skills
- Gitlab
- Writing documentation
- MarkdownHackweek 2023Gabagaba@torproject.orgGabagaba@torproject.org2023-11-09https://gitlab.torproject.org/tpo/team/-/issues/221Sponsor 9 Report2023-11-30T18:35:57ZGabagaba@torproject.orgSponsor 9 ReportWrite report for sponsor 9 for previous phase (July 2022 to June 2023).Write report for sponsor 9 for previous phase (July 2022 to June 2023).Gabagaba@torproject.orgGabagaba@torproject.org2023-11-16https://gitlab.torproject.org/tpo/network-health/onbasca/-/issues/169Check `Token` header instead of `Cookie` header from rdsys2023-11-06T08:36:47ZjugaCheck `Token` header instead of `Cookie` header from rdsysIn #162 we decided that onbrisca authorizes the requests via a token in `Cookie` header, however tpo/anti-censorship/rdsys#174 doesn't set it with the `Cookie` header but the `Token` and `Authorization` headers. Therefore onbrisca has al...In #162 we decided that onbrisca authorizes the requests via a token in `Cookie` header, however tpo/anti-censorship/rdsys#174 doesn't set it with the `Cookie` header but the `Token` and `Authorization` headers. Therefore onbrisca has already been changed in production to look at these header instead to authorize the request.jugajugahttps://gitlab.torproject.org/tpo/team/-/issues/219Budget for Translations into Turkmen2024-01-16T13:08:32ZGabagaba@torproject.orgBudget for Translations into TurkmenWe need to write the budget and scope to request Localization Lab for the translation for turkmen.
cc @emmapeelWe need to write the budget and scope to request Localization Lab for the translation for turkmen.
cc @emmapeelGabagaba@torproject.orgGabagaba@torproject.orghttps://gitlab.torproject.org/tpo/core/torspec/-/issues/223Convert specifications to mdbook2023-10-26T14:20:06ZNick MathewsonConvert specifications to mdbookPer [proposal 345](https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/345-specs-in-mdbook.md), I want to convert our specifications to markdown and render them in mdbook.
The end result of the migration will be that:...Per [proposal 345](https://gitlab.torproject.org/tpo/core/torspec/-/blob/main/proposals/345-specs-in-mdbook.md), I want to convert our specifications to markdown and render them in mdbook.
The end result of the migration will be that:
* The torspec repository looks more [like this](https://gitlab.torproject.org/nickm/torspec/-/tree/spec_conversion?ref_type=heads).
* There is rendered torspec website looks more [like this](https://people.torproject.org/~nickm/volatile/mdbook-specs/index.html).
It looks like [spec.tpo](https://spec.torproject.org) is available as a target for this rendering; @weasel (the current spec.tpo maintainer) has approved on IRC. There is a [TPA ticket](https://gitlab.torproject.org/tpo/tpa/team/-/issues/41348) open for the admin side of the issue, and I've gotten some helpful advice there too.
On this ticket I'll be tracking the actual details of doing the migration. I won't do anything final without talking to more people, though.
Next steps here are:
* [x] Decide on the new layout we want for torspec.git.
* [x] Decide on the URL layout we want for the new spec.tpo website.
- Should we have a landing page, or should the mdbook content **be** the landing page?
- Should we leave a spot for RFCs?
* [ ] Work on [the migration scripts](https://gitlab.torproject.org/nickm/torspec-converter) and their configuration (including where to break the sections), until they produces output we like, and it gives us the layout we want.
* [ ] Develop the CI process as needed to keep the site up to date.
- Probably, publish to gitlab pages at first.
* [ ] Figure out whose approval we need for this, and see what they think.
* [ ] Write documentation as needed to explain how to edit the spec.
* [ ] Decide how to maintain redirects and permalinks.
After we've done this stuff I think we are ready to start the migration.Nick MathewsonNick Mathewsonhttps://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/42154Empty the clipboard on browser shutdown only if content comes from private br...2023-11-07T16:50:23Zcypherpunks1Empty the clipboard on browser shutdown only if content comes from private browsing windowsThe user may not expect or need this feature (#42019) and maybe it should additionally be disabled for non-private browsing mode.
One issue is that the clipboard is emptied even if its contents did not originate from a browser.The user may not expect or need this feature (#42019) and maybe it should additionally be disabled for non-private browsing mode.
One issue is that the clipboard is emptied even if its contents did not originate from a browser.ma1ma1