Merge Process
Network team merge process, setup instructions, and notes.
This page explains how to merge patches into Tor. For the rules about when to merge patches into Tor, see the MergePolicy document.
How do you do a Merge?
Configuring your Environment
Master and Backport Merges
1. Make sure you have the git hooks installed
You can find the hooks in tor/scripts/git/. They need to be copied into tor/.git/hooks, without the extension:
- tor/scripts/git/post-merge.git-hook -> .git/hooks/post-merge
- (repeat for the other hooks)
Q: How do I keep the git hooks up to date? A: Install the post-merge (post pull) hook, so git logs a message when the hooks are updated in master. (See #29588.)
2. Clone and configure your git repository
You might find it easier to use a separate git repository for merges and regular coding.
Automatic Setup
Set TOR_FULL_GIT_PATH in your .profile, and run git-setup-dirs.sh.
(This script sets up backport worktrees as well: you can just ignore them.)
Manual Setup
Configure your repository with:
- git-rw.torproject.org:tor.git as "origin"
and
$ git remote add tor-gitlab https://gitlab.torproject.org/tpo/core/tor.gitPiece of advice here. Because git default behavior of "git push" is to use "origin", then mistake can happen where one of us can push a personal branch to our upstream tor.git.
For that I nullify the "pushurl" of origin with:
[remote "origin"]
fetch = +refs/heads/*:refs/remotes/origin/*
url = https://git.torproject.org/tor.git
pushurl = "Don't push personal branches to origin!"To push now, I created an "upstream" remote so when I push commits upstream, I have to explicitly type in:
$ git push upstream <branch>(The git-push-all.sh allows you to specific the name of the "upstream" branch).
To add the upstream remote properly:
$ git remote add upstream https://git.torproject.org/tor.git
$ git remote set-url --push upstream ssh://git@git-rw.torproject.org/tor.gitMake sure you have MRs from tor-gitlab, in your gitconfig:
[remote "tor-gitlab"]
fetch = +refs/merge-requests/*/head:refs/remotes/tor-gitlab/merge-requests/*
fetch = +refs/merge-requests/*/head:refs/remotes/tor-gitlab/mr/*
pushurl = No_Pushing_To_GitLab 3. Make sure you have merge permissions
Set up SSH keys using Tor's LDAP:
Check your access:
$ ssh git@git-rw.torproject.org | grep WFor more details, see org/operations/services/git.torproject.org.
Backport Merges Only
4. Make sure you have the git scripts installed
You can find the scripts in tor/scripts/git/. They need to be copied into your $PATH:
- tor/scripts/git/git-pull-all.sh -> ~/bin/git-pull-all.sh
- (repeat for the other scripts)
You will need to customise these scripts to your own environment, using the environmental variables listed in the script's --help.
These scripts require a recent bash that supports arrays.
5. Make sure you are using worktrees
You might find it easier to use a separate git repository for merges and regular coding.
Automatic Setup
Set TOR_FULL_GIT_PATH in your .profile, and run git-setup-dirs.sh.
Manual Setup
To use the git-* scripts, you need worktrees. Here is an example on how to do it properly for let say the "maint-0.3.5" branch and your worktrees are in "tor-wkt" (relative to tor.git/ you have):
$ git worktree add ../tor-wkt/maint-0.3.5 origin/maint-0.3.5
$ cd ../tor-wkt/maint-0.3.5
$ git checkout -b maint-0.3.5
$ git branch --set-upstream-to=origin/maint-0.3.5 maint-0.3.5So then now within the worktree, you can simply do "git pull" and it will always fetch the latest from origin/maint-0.3.5 which is important because the git-pull-all.sh script requires that. These worktrees need to follow upstream at all times.
Basically, a maintainer should do the above for all maintained branches (including releases-*).
New Maint and Release Branches
Make sure that your git scripts are up to date with the new branches!
Automatic Setup
Set TOR_FULL_GIT_PATH in your .profile, and run git-setup-dirs.sh -u.
Manual Setup
Set up the new maint-* and release-* worktrees, using the instructions in step 5 above.
Merging a Branch
Master Merges
0. Make sure your environment is set up
- git hooks
- tor-gitlab custom remote
1. Make sure you have the latest version of master checked out in your git directory
$ git fetch upstream2. Make sure that you have the latest version of the branch you're merging
$ git fetch tor-gitlab3. Merge the branch
$ git merge tor-gitlab/mr/<NUM>5. Push to main
$ git push upstream main6. Update the ticket, and wait for CI
See "Backport Merges" Steps 8 & 9 for details.
Backport Merges
0. Make sure your environment is set up
- git hooks
- tor-gitlab custom remote
- git scripts
- worktrees
1. Make sure you have the latest version of all supported branches checked out in your worktrees
$ git-pull-all.sh2. Make sure that you have the latest version of the branch you're merging
You can get the supported branches and gitlab MRs using:
$ git-pull-all.sh3. Check all the things you're supposed to check
4. Go into the earliest target directory where you are merging today, and merge the branch
$ cd maint-0.3.5
$ git merge tor-gitlab/mr/<NUM>5. If the merge isn't clean, ask the patch author or a stable maintainer what to do
6. Run git-merge-forward.sh and verify that there are no conflicts
The merge-forward script will stop after any conflicts.
7. If the merge isn't clean, ask the patch author or a stable maintainer what to do
You can go in to the worktree and fix the conflict yourself, if you can work out how. Let the patch author know on the ticket, and ask them to check your merge.
8. Push all the branches
$ git-push-all.sh9. Update the ticket
If you are merging a patch into the earliest series to which it applies, close the ticket after you merge, and say where you merged it. If you are merging a patch that is a candidate for backport, move the ticket into the earliest open milestone to which it applies.
9. Wait for the CI to finish on master
If the CI fails due to practracker errors, update the practracker exceptions file, and push to master again. (practracker fixes don't need review.) Ask the person who wrote the code to fix the exception.
Make a note on the trac ticket for the PR that caused the error.
Merging Different Branches into Different Releases
Follow the "Backport Merges" steps 3-5 for each branch you need to merge.
Finding Branches to Merge
- Work out all the maint branches you are merging pull requests into today:
- if you're doing a mainline merge, the maint branches might be:
- the alpha maint branch (if there is one), or
- master (if you'll backport to alpha later)
- if you're doing a backport merge, the maint branches might be:
- the oldest LTS release, or
- newer supported/LTS releases (but not the mainline releases)
- if you're doing a security fix merge:
- merge into all maint branches with the vulnerability
-
Work out the pull request for the earliest maint branch: a. If there is a pull request based on that maint branch, use that pull request b. Otherwise, check for a pull request for the next earliest version of Tor, and use that pull request c. Repeat b. with the Nth earlier version of Tor
-
Work out the maint branches for each other pull request: a. If there is a pull request based on that maint branch, use that pull request b. Otherwise, don't merge any pull request into that maint branch
- the pull request for an earlier version should merge forward cleanly
To simplify this process, we usually:
- Merge the change to master
- Wait for testing in an alpha release
- Merge the change to all other versions
Doing the Merges
Here's how you do the merges:
- Find the earliest maint branch you want to merge into today, and the pull request for that maint branch
- Change to the directory for the maint branch
- Merge the pull request in to the maint branch
- Repeat from 2 for the next pull request and its maint branch
- Once all the pull requests are merged, merge all branches forward
Dealing with Merge Conflicts
If there are any merge conflicts, stop! Don't push anything! Reset your merge directories to the current branches on torproject.org. Ask for help on the ticket. Include details of the merge conflict.