A DEVELOPMENT NOTE:
===================

Developing the reproducible builds environment for Tor Browser with
gitian/tor-browser-bundle is deprecated. Recently, we switched to
rbm/tor-browser-build. See:
https://trac.torproject.org/projects/tor/ticket/23213. To get started with the
new system, check out
https://git.torproject.org/builders/tor-browser-build.git and follow the
instructions in the README file.

-------------------

QuickStart:

 On an Ubuntu 14.04+/Debian Wheezy+ machine or VM, run:

 $ make

 This will check all of your prerequisites and tell you what you need to
 do to get started to build the stable TBB.

 If everything checks out OK, it will begin downloading dependencies, and then
 start the build process to produce localized Linux bundles, followed by
 Windows bundles, followed by Mac bundles.

 To check your build results against the official builders, run:

 $ make match

 By default, the Makefile wraps input downloads in 'torsocks'. This is done
 to allow unofficial secret verifiers to remain secret by default. To download
 inputs without torsocks, run:

 $ make TORSOCKS=

 By default, 'make' will also clear out any previous partial builds. See the
 "Partial Rebuilds" section for information on performing partial rebuilds
 without erasing previously built components.

 If you would also like to download fresh copies of your inputs before a
 build, run 'make distclean' (note that timestamps and versions of downloaded
 source input files are checked every time you run 'make' by default so
 distclean should not normally be needed).

 To set the number of make processes and virtual CPUs to use inside the VMs
 to four (the default is two), use:

 $ export NUM_PROCS=4

 To set the amount of RAM to use inside the VMs to 8000 MiB (the default is
 4000 MiB), use:

 $ export VM_MEMORY=8000

Detailed Explanation of Scripts:

 This directory is a wrapper around our modified version of Gitian, and has
 several helper scripts to make things easier.

 0. Makefile: The main Makefile. It has the following commands:
     - all: The default. It calls the 'clean', 'prep' and 'build' rules
     - prep: Check OS prerequisites and download source dependency inputs
     - build: Build localized bundles for Linux, Windows, and Mac
     - clean: Remove prior partial build stages (see 'Partial Rebuilds' below
       for the usage of clean-* commands)
     - vmclean: Remove VM base images
     - distclean: Remove source dependency inputs, and run 'clean' and 'vmclean'
     - sign: Sign your build output and uploads it to people.torproject.org
     - match: Check your build output against public signed hashes
     - hash: Create the SHA256 sums
     - incrementals: Create the incremental update (.mar) files
     - update_responses: Create .xml and .htaccess files for the updater
     - signmars: Create the signatures for the update (.mar) files
    To build beta/alpha/nightly bundles, alternate targets are provided:
     - nightly: The equivalent to the 'all' rule for nightly packages
     - alpha: The equivalent to the 'all' rule for alpha packages
     - prep-nightly: The equivalent to the 'prep' rule for nightly packages
     - prep-alpha: The equivalent to the 'prep' rule for alpha packages
     - build-nightly: The equivalent to the 'build' rule for nightly packages
     - build-alpha: The equivalent to the 'build' rule for alpha packages
     - sign-nightly: The equivalent to the 'sign' rule for nightly packages
     - sign-alpha: The equivalent to the 'sign' rule for alpha packages
     - match-nightly: The equivalent to the 'match' rule for nightly packages
     - match-alpha: The equivalent to the 'match' rule for alpha packages
     - hash-nightly: The equivalent to the 'hash' rule for nightly packages
     - hash-alpha: The equivalent to the 'hash' rule for alpha packages
     - incrementals-alpha: The equivalent to the 'incrementals' rule for
       alpha packages
     - update_responses-alpha: The equivalent to the 'update_responses' rule
       for alpha packages
     - signmars-nightly: The equivalent to the 'signmars' rule for nightly packages
     - signmars-alpha: The equivalent to the 'signmars' rule for alpha packages

 1. check-prerequisites.sh: This script checks if your system is capable of
    running Gitian, and if it is not, it tells you what you need to do.
    It is called by 'make prep'.

 2. fetch-inputs.sh: This script fetches all of our source dependencies from
    the Internet and downloads them into ../../gitian-builder/inputs.
    After you run this script, you should be able to complete the rest of your
    build offline.

 3. versions: This scriptlet contains version definitions for the source
    dependencies, the git tags to use, and the Tor Browser release version.

 4. verify-tags.sh: This script verifies the signatures on git tags from
    the versions file. It is only run if VERIFY_TAGS is set in the versions
    file.

 5. descriptors: In the descriptors directory, you will find the Gitian
    descriptors for Linux, Windows, and Mac. There are five descriptors for
    each platform: One to build the utilities needed by other descriptors,
    one to build Tor, one to build Tor Browser, one to build the Pluggable
    Transports and one to bundle everything together. Each descriptor is run
    from a fresh VM.

 6. mkbundle-linux.sh: This script is a wrapper around Gitian's gbuild to call
    the appropriate descriptors to build 32 and 64 bit Linux bundle. It also will
    create build VM images for you if you haven't done that before.

 7. mkbundle-windows.sh: This script is a wrapper around Gitian's gbuild to
    call the appropriate descriptors to build a Windows bundle. It also will
    create build VM images for you if you haven't done that before.

 8. mkbundle-mac.sh: This script is a wrapper around Gitian's gbuild to
    call the appropriate descriptors to build a 32bit Mac bundle. It also
    will create build VM images for you if you haven't done that before.

 9. record-inputs.sh: This script records all of the hashes and bundle
    inputs for inclusion in the 'Docs/sources' subdirectory of the bundles
    themselves.

 10. hash-bundles.sh: This script generates a 'sha256sums-unsigned-build.txt'
     file in sorted, reproducible order.

 11. check-match.sh: This script checks your 'sha256sums-unsigned-build.txt'
     file against any signed, published builds.

 12. upload-signature.sh: This script signs and uploads your
     'sha256sums-unsigned-build.txt' file (for use if you are an official
     builder).
 13. signmars.sh: This script generates the signatures on the update (.mar)
     files. It expects an nssdb directory, containing the key, in the same
     directory (i.e. tor-browser-bundle/gitian where it is located, too) and
     a certificate with "marsigner" as nickname. Both the default nssdb
     directory and the default certificate nickname can get overwritten by
     exporting 'NSS_DB_DIR' and 'NSS_CERTNAME' pointing to the directory and
     the nickname to be used instead.

Partial Rebuilds:

  The mkbundle shell scripts each have five Gitian stages:
    1. Build support utilities (gitian-utils.yml)
    2. Tor (gitian-tor.yml)
    3. Tor Browser (gitian-firefox.yml)
    4. Pluggable Transports (gitian-pluggable-transports.yml)
    5. Addon and bundle package assembly (gitian-bundle.yml)

  Each stage leaves build artifacts in ../../gitian-builder/inputs/ for use
  in later stages. If the artifact files for a given stage are present, that
  stage is skipped by a 'make resume'.

  No other dependency checking on build stages is performed.

  If you wish to rebuild only a given stage, there are several 'make clean-'
  targets to remove the artifacts for that stage. Running a subsequent
  'make resume' after that will rebuild only those targets that have been
  cleaned, and then re-package everything again as well.

  Note that if any of the source inputs, or the Ubuntu buildchain tools, or
  the build tools' dependencies upgrade, your Tor and Tor Browser inputs will
  no longer match those produced by someone else's build from a fresh build
  that downloaded those newer input, dependency, and toolchain packages.

  This means for official builds, it is a good idea to always run a full
  'make clean' first, or just run 'make'.


Known Issues and Quirks:

  1. Sometimes, fresh VM installs can fail. If your build hangs while trying
     to connect to the target or during the SSH banner, it might be worth
     running 'make vmclean && make resume' to clean your VM images, but still
     resume the build process as the component you left off at.  Remember,
     'make' by itself runs a clean rule that will wipe your previous builds by
     default.

     If you have already completed a Linux build, you may run into situations
     where 'make vmclean' causes the rebuild of two VMs in a row.. This might
     trigger weird bugs in python-vm-builder.. To rebuild only one set of VMs,
     use either 'rm ../../gitian-builder/*precise*' (to remove the Windows/Mac
     VMs) or 'rm ../../gitian-builder/*wheezy*' (to remove the Linux VMs).

     You probably want to make sure you have no stray qemu processes before
     rebuilding the VMs or starting a new build, too. 'killall qemu-kvm' is
     your friend.

     Once you get a working set of base VMs (in ../../gitian-builder/base-*),
     you should probably copy them somewhere safe if you want to avoid VM
     creation hangs in the future (or help us write a wrapper script that
     tests VMs and re-runs the VM creation step if they don't boot).

  2. If you use git branches for any repos instead of tags (for example, for
     a development or nightly build), fresh commits will need to be
     merged manually (or better git-fu is needed in ./fetch-inputs.sh, as
     it is currently meant to deal with tags only).

  3. Running more than one instance of Gitian at a time is not supported.

  4. Related: If you perform a fresh Gitian checkout for purposes of
     verification, be sure to kill any stray qemu VM processes before starting
     the new build (because the new Gitian checkout will not have the PID or
     SSH keys of the previous instances' VM, and VM startup will either hang
     or prompt you for an SSH password).


Diagnosing Problems:

  During a running build, you can tail logs in real time in
  ../../gitian-builder/var/*.log

  Upon failure, logs of any failed component builds are relocated to
  ../../gitian-builder/*fail*.log. This is to prevent subsequent builds
  from destroying failure information.

