Skip to content
Snippets Groups Projects
Unverified Commit 2b13c342 authored by boklm's avatar boklm
Browse files

Bug 40049: processes/ReleaseProcess is replaced by gitlab issue templates

parent 2b940775
Branches
No related tags found
1 merge request!45Bug 40049: processes/ReleaseProcess is replaced by gitlab issue templates
Show whitespace changes
Inline Side-by-side
processes/ReleaseProcess
+ 5
− 359
View file @ 2b13c342
# Magical Tor Browser Release Process Incantations
#
# "May this part of our job one day be replaced by a small shell script"
# The Tor Browser Release Process is now documented as gitlab issue templates:
# - https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/main/.gitlab/issue_templates/Release%20Prep%20-%20Tor%20Browser%20Stable.md
# - https://gitlab.torproject.org/tpo/applications/tor-browser-build/-/blob/main/.gitlab/issue_templates/Release%20Prep%20-%20Tor%20Browser%20Alpha.md
#
#####################
### Prepare build ###
#####################
#. Tag any relevant component versions.
# In particular: before tagging the tor-browser tag, the final code from
# Torbutton needs to get included and in order to avoid unnecessary commit
# inflation the commit for bumping the git submodule should be a fixup commit
# to the one with "Bug 25013: Add torbutton as a tor-browser submodule" in
# its commit message.
#. Update changelog and relevant config files in tor-browser-build.
cd tor-browser-build
vim projects/firefox/config
vim ChangeLog.txt
vim rbm.conf
#. Tag a build tag in tor-browser-build.
make signtag-release # or `make signtag-alpha` for an alpha build
#. Push tag and version to tor-browser-build.git. In case of doing a stable
# release with a maintenance branch use that one instead of `master`.
torsocks git push origin master:master
torsocks git push origin --tags
#####################
### Build ###
#####################
#. Build and generate incremental MAR files.
git tag -v tbb-$TORBROWSER_VERSION-buildN
git checkout tbb-$TORBROWSER_VERSION-buildN
make && make incrementals-release # `make alpha && make incrementals-alpha`
#. Compare the SHA256 sums of the bundles and MAR files with an independent
# builder.
sha256sum tor-browser-build/release/unsigned/$TORBROWSER_BUILDDIR/sha256sums-unsigned-build.txt
sha256sum tor-browser-build/release/unsigned/$TORBROWSER_BUILDDIR/sha256sums-unsigned-build.incrementals.txt
#. If the sums match (download and) upload the bundles to your build dir on
# people.torproject.org. Fix permissions.
chmod 755 $TORBROWSER_BUILDDIR
chmod 644 $TORBROWSER_BUILDDIR/*
chmod 644 $TORBROWSER_BUILDDIR/.htaccess
torsocks ssh people.torproject.org "mkdir ~/public_html/builds/${TORBROWSER_BUILDDIR}"
torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR
#. (Optional): Upload your binaries to people using partial rsync over old version
torsocks ssh people.torproject.org "mv ~/public_html/builds/${TORBROWSER_VERSION}-build1 ~/public_html/builds/$TORBROWSER_BUILDDIR"
torsocks rsync -avP $TORBROWSER_BUILDDIR/ people.torproject.org:public_html/builds/$TORBROWSER_BUILDDIR
#. Distribute build to tor-qa@lists.torproject.org
# XXX: Currently manual email with link to candidate build, important changes,
# and changelog.
# For stable releases put tails-dev@boum.org into Cc.
#######################
### Android signing ###
#######################
#. The Android signing is done separately from the desktop signing. To
# do the Android signing you'll need to boot from a Tails usb stick,
# with persistent storage enabled.
#. Install the openjdk-11-jdk-headless package
apt install openjdk-11-jdk-headless
#. Make sure you have the android keys, for example in the
# ~/Persistent/android-keys directory. This directory should include
# the tba_alpha.p12 and tba_release.p12 files.
#. Clone tor-browser-build, and checkout the tag for the release
git tag -v tbb-$TORBROWSER_VERSION-buildN
git checkout tbb-$TORBROWSER_VERSION-buildN
#. Update set-config.tbb-version and set-config.android-signing
cd tools/signing
vim set-config.tbb-version
vim set-config.android-signing
#. Run the android-signing script. This script will download the
# unsigned apks from the `pkgstage` machine (where `do-all-signing`
# is run), signed them, and upload back the signed apks to the
# `pkgstage` machine. It should be done before the hash_signed_bundles.sh
# step.
./android-signing
##########################
### Signing and upload ###
##########################
#. You need a `pkgstage` machine to store the bundles during the signing
# process. This machine should be:
# - secure (you will ssh to the signing machines and staticiforme
# from there)
# - with good bandwidth (you will download/upload bundles from there)
#
# All steps from the `Signing and upload` are run from the `pkgstage`
# machine unless mentioned otherwise.
#
# The signing scripts are located in the tor-browser-build/tools/signing
# directory.
#
# Alternatively, you can run the script `./do-all-signing`, which will
# do all the signing steps automatically, until the static-update-component
# commands on staticiforme. Also the script does not currently handle
# removing the old versions on staticiforme, so you have to do that
# manually. The script is also not running `./macos-signer-proxy` which
# needs to be started and stopped manually.
#. Checkout the tor-browser-build.git commit corresponding to the
# tor browser version you need to sign/publish.
cd tor-browser-build
git tag -v tbb-$TORBROWSER_VERSION-buildN
git checkout tbb-$TORBROWSER_VERSION-buildN
#. Change to the tor-browser-build/tools/signing directory. All following
# commands are run from this directory.
cd tor-browser-build/tools/signing
#. Set hosts information.
vim set-config.hosts
#. Set tor browser version information. You should set tbb_version,
# tbb_version_build, tbb_version_type. This should match the information
# in rbm.conf.
vim set-config.tbb-version
#. Download the tor browser build to the `pkgstage` machine from your
# build machine (configured in set-config.hosts).
./sync-builder-unsigned-to-local-signed
#. Codesign the macOS dmg files.
#
# Updload the tor browser dmg files to `macos-signer`
./sync-macos-local-to-macos-signer
# Upload updated signing scripts to `macos-signer`. The scripts are
# located in the `signing-release` or `signing-alpha` directory on
# `macos-signer`.
./sync-scripts-to-macos-signer
# Run proxy for `macos-signer`. You may need to kill an old proxy
# process on `macos-signer` if it was still running.
./macos-signer-proxy
# Enable networking on the `macos-signer`.
macos-signer$ networksetup -setsecurewebproxystate Ethernet on
# Sign the bundles. On `macos-signer`.
# (replace signing-release with signing-alpha for an alpha release)
macos-signer$ ~/signing-release/macos-signer-gatekeeper-signing
# Notarize the bundles. On `macos-signer`.
# (replace signing-release with signing-alpha for an alpha release)
macos-signer$ ~/signing-release/macos-signer-notarization
# Staple the bundles. On `macos-signer`.
# (replace signing-release with signing-alpha for an alpha release)
macos-signer$ ~/signing-release/macos-signer-stapler
# Disable networking on the `macos-signer`.
macos-signer$ networksetup -setsocksfirewallproxystate Ethernet off
# Download the stapled bundles to `pkgstage`.
./sync-macos-signer-stapled-to-macos-local-stapled
#. Regenerate macOS DMG files from stapled zip files.
./gatekeeper-bundling.sh
#. Regenerate macOS MAR files from code signed dmg files.
./dmg2mar
#. Sync bundles to `linux-signer`.
./sync-local-to-linux-signer
#. Upload updated signing scripts to `linux-signer`. The scripts are
# located in the `signing-release` or `signing-alpha` directory on
# `linux-signer`.
./sync-scripts-to-linux-signer
#. Sign the MAR files. On `linux-signer`.
# (replace signing-release with signing-alpha for an alpha release)
linux-signer$ ~/signing-release/linux-signer-signmars
#. Authenticode signing of exe files. On `linux-signer`.
linux-signer$ ~/signing-release/linux-signer-authenticode-signing
#. Authenticode timestamping.
./sync-linux-signer-to-local
./authenticode-timestamping.sh
#. Create sha256sums-signed-build files
./hash_signed_bundles.sh
#. Upload sha256sums-signed-build and updated exe files to `linux-signer`.
./sync-local-to-linux-signer
#. Gpg signing.
./linux-signer-gpg-sign
#. Fetch signatures on unsigned sha256sums from other builds
./download-unsigned-sha256sums-gpg-signatures-from-people-tpo
#. Remove old builds on `staticiforme`
# IMPORTANT: Remove the oldest version in a series in case there is more
# than 1 available on dist.torproject.org before proceeding
# XXX: TORBROWSER_VERSION_OLDEST needs to be set
staticiforme$ rm -rf /srv/dist-master.torproject.org/htdocs/torbrowser/$TORBROWSER_VERSION_OLDEST
staticiforme$ static-update-component dist.torproject.org
#. Upload new build to `staticiforme`
./sync-local-to-staticiforme
#. Check diskspace available on cdn.tpo
# We currently have enough disk space to host two alpha and stable
# releases. However with the size of each Tor Browser release increasing,
# it may become necessary to increase disk space. The server hosting
# the files for cdn.tpo is savii.tpo and its disk usage can be monitored
# by going to https://grafana.torproject.org/d/Z7T7Cfemz/node-exporter-full
# and selecting the hosts: web-fsn-02.torproject.org, web-cymru-01.torproject.org,
# web-fsn-01.torproject.org, and cdn-backend-sunet-01.torproject.org
#. Remove the oldest *.mar files from cdn.tpo to save space
staticiforme$ rm -rf /srv/cdn-master.torproject.org/htdocs/aus1/torbrowser/$TORBROWSER_VERSION_OLDEST
staticiforme$ static-update-component cdn.torproject.org
#. Sync files to dist.tpo and cdn.tpo mirrored web servers
# (replace signing-release with signing-alpha for an alpha release)
# It is a good idea to run the static-update-component inside screen
# or tmux so that the upload continues if your connection to
# staticiforme is closed.
$ ./sync-scripts-to-staticiforme
staticiforme$ ~/signing-release/staticiforme-prepare-cdn-dist-upload
staticiforme$ static-update-component cdn.torproject.org && \
static-update-component dist.torproject.org
#. Make sure we really built from the proper Mozilla build tag by consulting
# the respective ESR release branch (for a good overview for ESR78 see
# https://hg.mozilla.org/releases/mozilla-esr78/graph/). For the platforms following
# rapid release (only Android, currently), consult the beta repo
# (https://hg.mozilla.org/releases/mozilla-beta/graph/) or the release repo
# (https://hg.mozilla.org/releases/mozilla-release/graph/)
#. Update website's torbrowser versions file in the website git
cd tpo
torsocks git pull origin
# Update `win32` in the `torbrowser-stable` section as well if we
# include a new stable tor version (called the Windows Expert Bundle
# on the website). See: #14152.
vim databags/versions.ini
git commit databags/versions.ini -m "Add new Tor Browser version"
torsocks git push origin master:master
cd ..
# Check build success/failure:
# https://jenkins.torproject.org/job/lektor-website-tpo-translation/
# https://jenkins.torproject.org/job/lektor-website-tpo-translation-install/
#. Add new locales to the download page
# If this release is introducing new locales, add them to the
# databags/download-alternatives.ini file (for a stable release) or
# the databags/download-alternatives-alpha.ini file (for an alpha
# release).
cd tpo
torsocks git pull origin
vim databags/download-alternatives.ini # or databags/download-alternatives-alpha.ini
git commit databags/download-alternatives.ini -m "Add new Tor Browser locales"
torsocks git push origin master:master
cd ..
#. Create blog post from changelog
# Edit set-config.blog to set you local blog directory.
# Don't forget to link to Mozilla's security advisories if this is a security
# update.
./create-blog-post
#. Check whether the .exe files got properly signed and timestamped
# Point OSSLSIGNCODE to your osslsigncode binary
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
OSSLSIGNCODE=/path/to/osslsigncode
../../../tools/authenticode_check.sh
popd
#. Check whether the MAR files got properly signed
# Point NSSDB to your nssdb containing the mar signing certificate
# Point SIGNMAR to your signmar binary
# Point LD_LIBRARY_PATH to your mar-tools directory
pushd tor-browser-build/${channel}/signed/$TORBROWSER_VERSION
NSSDB=/path/to/nssdb
SIGNMAR=/path/to/mar-tools/signmar
LD_LIBRARY_PATH=/path/to/mar-tools/
../../../tools/marsigning_check.sh
popd
#. Generate and upload update_responses to staticiforme
./upload-update_responses-to-staticiforme
#. Enable update. On `staticiforme`.
# (for alpha release, use deploy_update_responses-alpha.sh instead)
staticiforme$ ~/deploy_update_responses-release.sh
#. Clean linux-signer and macos-signer
./finished-signing-clean-linux-signer
./finished-signing-clean-macos-signer
# Upload APKs to Google Play
Log into https://play.google.com/apps/publish
Select correct app (Tor Browser or Tor Browser Alpha)
Under left-side navigation bar, select "Production" under "Release"
Select "Create new release" button at upper right-side
Under "App bundles and APKs" section, "Upload" each signed APK
After uploading APKs:
- The "Release Name" should be automatically filled with the Firefox version
- Next to "Release notes" click "Copy from a previous release"
- Select the previous release and adjust the blog post url for this release
- Save, review, and configure rollout percentage at the bottom
- Use 25% rollout when publishing a scheduled update (without a security driver)
- Use 100% rollout when publishing an unscheduled update or security-driven release
Roll out.
Note, you may receive three warning messages about:
1) app is larger than it is necessary (android app bundle),
2) missing deobfuscation file for better crash analysis
3) missing debug symbol file
These warnings are expected and should be accepted.
See below for updating the rollout percentage.
# Release on F-Droid
Publication on F-Droid via the Guardian Project's repository should be
automatic. Hans is responsible for maintaining this automation.
# Update rollout percentage
After 24 hours, check the Release Daskboard:
- "Release"->"Production" and select the "Release Dashboard" tab
If the Release Dashboard does not show any concerning trends (significant
increase in crashes or ANRs (Application Not Responding)), then continue on
to the next paragraph and increase rollout from 25% to 100%. Otherwise
consider halting rollout as described in the `RollingBackUpdate` process.
Select the "Releases" tab on the "Production" page. The current released
version should indicate 25% rollout: "Available to 25% of users on Google Play"
On the right-side of the "View release details" button of the release there
should be a button labeled "Manage rollout" with a down-arrow. Clicking on
that button should show two options:
- Update rollout
- Halt rollout
Select "Update rollout" and increase to 100% and click "Update". The change
should be immediately implemented and the "Manage rollout" button disappears.
#. Write an email to tor-announce in case this release is concerned with getting
# a stable version out. Using the contents of the blog entry should do it.
# See also the open release prep issues:
# https://gitlab.torproject.org/groups/tpo/applications/-/issues/?sort=updated_desc&state=opened&label_name%5B%5D=Release%20Prep&first_page_size=20
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Please to comment