Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-16T01:24:55Zhttps://gitlab.torproject.org/legacy/trac/-/issues/22341Make sure to pick up zstd+lzma support for tor in Tor Browser2020-06-16T01:24:55ZGeorg KoppenMake sure to pick up zstd+lzma support for tor in Tor BrowserWe might need to adapt our descriptors to make sure tor in Tor Browser is built with zstd+lzma support as well.
This is the parent ticket and the work, if needed, is done in child tickets.We might need to adapt our descriptors to make sure tor in Tor Browser is built with zstd+lzma support as well.
This is the parent ticket and the work, if needed, is done in child tickets.https://gitlab.torproject.org/legacy/trac/-/issues/21565Clean up hardening wrapper flags used in tor-browser-build2020-06-16T01:24:53ZGeorg KoppenClean up hardening wrapper flags used in tor-browser-buildWe set `export DEB_BUILD_HARDENING=1` to activate GCC hardening flags but we export other *HARDENING flags, too. The latter seems to be unnecessary as the former should already be activating all flags available. We should clean that up.We set `export DEB_BUILD_HARDENING=1` to activate GCC hardening flags but we export other *HARDENING flags, too. The latter seems to be unnecessary as the former should already be activating all flags available. We should clean that up.https://gitlab.torproject.org/legacy/trac/-/issues/21448Identify what build flags we should be using for security, and use them2020-06-16T01:24:53ZArthur EdelsteinIdentify what build flags we should be using for security, and use themI think we may be able to add some configure/compiler/linker flags in Tor Browser that can improve security without many downsides. Let's figure out what those are and add them.I think we may be able to add some configure/compiler/linker flags in Tor Browser that can improve security without many downsides. Let's figure out what those are and add them.https://gitlab.torproject.org/legacy/trac/-/issues/21032Creating some public database of "reproduced builds"2020-06-16T01:24:52ZboklmCreating some public database of "reproduced builds"The process of checking that our builds have been reproduced by multiple people is currently mostly manual. In order to make this process easier, more automated (to be able to use it in the updater or some launcher) and possible to use a...The process of checking that our builds have been reproduced by multiple people is currently mostly manual. In order to make this process easier, more automated (to be able to use it in the updater or some launcher) and possible to use at a larger scale (checking that some large number of people reproduced a build), we could have some tool indexing the builds created by various people.
This could be done by adding the generation of some `buildinfo` files (similar to the Debian's buildinfo files) to our build process, containing important informations about the build, such as its inputs and outputs, and indexing them with their signatures in some database.
This database would contain the following types of builds or operations, signed by various builders:
- the build of a bundle from a git tag
- the creation of a signed mar file, from an unsigned mar (or the reverse operation)
- the creation of an OSX code-signed mar file, from an unsigned mar (or the reverse operation)
- the creation of an incremental mar file, from two full mar fileshttps://gitlab.torproject.org/legacy/trac/-/issues/20361Investigate CFI means for usage in Tor Browser2020-06-16T01:24:51ZGeorg KoppenInvestigate CFI means for usage in Tor BrowserChrome uses CFI for some parts in its code: https://www.chromium.org/developers/testing/control-flow-integrity and Mozilla is about to add CFI support to the build system (https://bugzilla.mozilla.org/show_bug.cgi?id=1302891). We should ...Chrome uses CFI for some parts in its code: https://www.chromium.org/developers/testing/control-flow-integrity and Mozilla is about to add CFI support to the build system (https://bugzilla.mozilla.org/show_bug.cgi?id=1302891). We should investigate whether we can leverage that at least in our hardened-builds as wellhttps://gitlab.torproject.org/legacy/trac/-/issues/20322SafeSEH support for mingw-w64 for Tor Browser on Windows2020-06-16T01:24:50ZbugzillaSafeSEH support for mingw-w64 for Tor Browser on WindowsNot only SEH, because of www.fuzzysecurity.com/tutorials/expDev/3.html
Even YASM can do it https://www.tortall.net/projects/yasm/manual/html/objfmt-win32-safeseh.htmlNot only SEH, because of www.fuzzysecurity.com/tutorials/expDev/3.html
Even YASM can do it https://www.tortall.net/projects/yasm/manual/html/objfmt-win32-safeseh.htmlhttps://gitlab.torproject.org/legacy/trac/-/issues/20254Update marsigning-check.sh to cope with signed OS X MAR files2020-06-16T01:25:57ZGeorg KoppenUpdate marsigning-check.sh to cope with signed OS X MAR filesNow that the fix for #19410 landed our `marsigning-check.sh` script can't check the correctness of our MAR file signatures easily anymore as we use the SHA256 sums of the unsigned MAR files currently. We should adapt the script.Now that the fix for #19410 landed our `marsigning-check.sh` script can't check the correctness of our MAR file signatures easily anymore as we use the SHA256 sums of the unsigned MAR files currently. We should adapt the script.https://gitlab.torproject.org/legacy/trac/-/issues/20135OSX: fail during bundling if any symlinks are used2020-06-16T01:24:48ZMark SmithOSX: fail during bundling if any symlinks are usedIn #19410, mcs said: Regarding the symlink issue, as a failsafe we could check to make sure no symlinks are used, but we would need to do that as we create the dmg.In #19410, mcs said: Regarding the symlink issue, as a failsafe we could check to make sure no symlinks are used, but we would need to do that as we create the dmg.https://gitlab.torproject.org/legacy/trac/-/issues/19182Find a better place for the Mac OS X SDK we need for our Tor Browser builds2020-06-16T01:24:48ZGeorg KoppenFind a better place for the Mac OS X SDK we need for our Tor Browser buildsRight now we host the OS X SDK we use for our Tor Browser builds on our infrastructure. We might want to find a better solution for that.Right now we host the OS X SDK we use for our Tor Browser builds on our infrastructure. We might want to find a better solution for that.https://gitlab.torproject.org/legacy/trac/-/issues/19181Firefox >= 48 ships with an ICU pre-compiled blob2020-06-16T01:24:47ZGeorg KoppenFirefox >= 48 ships with an ICU pre-compiled blobIn https://bugzilla.mozilla.org/show_bug.cgi?id=1239083 Mozilla implemented build changes that resulted in an ICU related binary being shipped in the source tree. It is a pre-compiled thing to avoid generating it twice e.g. in a cross-co...In https://bugzilla.mozilla.org/show_bug.cgi?id=1239083 Mozilla implemented build changes that resulted in an ICU related binary being shipped in the source tree. It is a pre-compiled thing to avoid generating it twice e.g. in a cross-compilation scenario. We should investigate whether we want to ship that blob.https://gitlab.torproject.org/legacy/trac/-/issues/18867Ship auto-updates for Tor Browser nightly channel2020-06-16T01:24:47ZGeorg KoppenShip auto-updates for Tor Browser nightly channelWe want to get more users to test our nightly builds. The main hurdle currently is the missing auto-update. This is the parent ticket that tracks this task.We want to get more users to test our nightly builds. The main hurdle currently is the missing auto-update. This is the parent ticket that tracks this task.https://gitlab.torproject.org/legacy/trac/-/issues/18326Creating incremental MAR files should not include Tor Browser version in meta...2020-06-16T01:24:45ZGeorg KoppenCreating incremental MAR files should not include Tor Browser version in meta dataWhile trying to build the incrementals for 5.5.2 on a machine that had older mar-tools I ended up with them not matching incrementals built on other machines. Upon investigation it turned out that the browser versions gets embedded in th...While trying to build the incrementals for 5.5.2 on a machine that had older mar-tools I ended up with them not matching incrementals built on other machines. Upon investigation it turned out that the browser versions gets embedded in the MAR files' metadata. There is no need for doing that actually.https://gitlab.torproject.org/legacy/trac/-/issues/18325We should not depend on rebuilding Firefox to get a new Tor Browser version out2020-06-16T01:24:44ZGeorg KoppenWe should not depend on rebuilding Firefox to get a new Tor Browser version outIn case we need to bump OpenSSL or think about just rebundling for a new release we need to rebuild Firefox too, currently, as the Tor Browser version is getting set there. We should find a way to avoid that.In case we need to bump OpenSSL or think about just rebundling for a new release we need to rebuild Firefox too, currently, as the Tor Browser version is getting set there. We should find a way to avoid that.https://gitlab.torproject.org/legacy/trac/-/issues/17382Using tor to download all build dependencies2020-06-16T01:24:43ZboklmUsing tor to download all build dependenciesThe current Tor Messenger build process does not use tor to download the various build dependencies. We should add an option (on by default) to do that and allow anonymous builders.The current Tor Messenger build process does not use tor to download the various build dependencies. We should add an option (on by default) to do that and allow anonymous builders.https://gitlab.torproject.org/legacy/trac/-/issues/17381Adapting the gitian/*.sh release scripts2020-06-16T01:24:43ZboklmAdapting the gitian/*.sh release scriptsThe various gitian/*.sh scripts will need to be adapted.The various gitian/*.sh scripts will need to be adapted.https://gitlab.torproject.org/legacy/trac/-/issues/17379Using the same build process in Tor Browser and Tor Messenger2020-06-16T01:24:42ZboklmUsing the same build process in Tor Browser and Tor MessengerThe current Tor Browser build process is using gitian to run the build inside virtual machines, with some shell scripts to download and verify dependencies.
Tor Messenger is using a different system to do that.
To make maintenance easi...The current Tor Browser build process is using gitian to run the build inside virtual machines, with some shell scripts to download and verify dependencies.
Tor Messenger is using a different system to do that.
To make maintenance easier, we should try to use the same tools in both projects.
I think one of the main advantages of the way we are building in tor-messenger is that we can do faster builds as we rebuild only what is necessary. All components are built separately, with all built components having an identifier in their filename that is a hash of all their dependencies (build script, commit hash, source tarballs,
distribution used in the container, additional packages installed). We only rebuild a component when its identifier has changed. We currently only have one branch on tor messenger, but with multiple branches (alpha, beta, nightly, etc ...) it would be possible to build the different branches from the same build repository and share the components that are identical.
An other advantage of splitting the build of each component is to be able to share them between different projects. For instance we are also building a Tor Mail bundle from the same repository and sharing some of the components. In the case of Tor Browser and Tor Messenger I think we could share the compilers. Components can also be built differently
depending on which project they are built for, for instance tor-launcher is patched to use a different control port for tor-mail and tor-messenger:
https://gitweb.torproject.org/tor-messenger-build.git/tree/projects/tor-launcher/controlport.patch.tmpl
https://gitweb.torproject.org/tor-messenger-build.git/tree/rbm.conf#n74
I will open child tickets to list all the tasks that need to be done if we want Tor Browser to use the same build process as Tor Messenger.https://gitlab.torproject.org/legacy/trac/-/issues/17336consider re-dzip'ing language packs2020-06-16T01:24:41ZMark Smithconsider re-dzip'ing language packsWhile doing some MAR file generation experiments, Kathy and I found that running re-dzip.sh on the Firefox language packs resulted in smaller incremental MAR files. Unfortunately, we did not make a note of the number of bytes saved. We s...While doing some MAR file generation experiments, Kathy and I found that running re-dzip.sh on the Firefox language packs resulted in smaller incremental MAR files. Unfortunately, we did not make a note of the number of bytes saved. We should repeat this experiment to determine whether it is worthwhile to re-dzip the language packs.https://gitlab.torproject.org/legacy/trac/-/issues/15710Consider enabling the BCJ filter when packaging bundles.2020-06-16T01:24:41ZYawning AngelConsider enabling the BCJ filter when packaging bundles.xz (and xz embedded) supports branch/call/jump filters that is supposed to reduce compressed executable size. The documentation hints that the compression ratio can end up being worse in certain situations, but as far as I can tell, it ...xz (and xz embedded) supports branch/call/jump filters that is supposed to reduce compressed executable size. The documentation hints that the compression ratio can end up being worse in certain situations, but as far as I can tell, it helps.
Numbers (tested with `tor-browser-linux64-4.5a5_en-US.tar.xz`):
* Current settings: 43667412 (1.0)
* `xz -9`: 43463244 (0.99)
* `xz --x86 --lzma2`: 42621816 (0.976)
* `xz --x86 --lzma2=preset=9`: 42402308 (0.971)
The "extreme" setting doesn't help the bundle compression enough to be worth it, and it would increase the already long build time even further.
The improvement isn't absolutely massive, but ~1 MiB is ~1 MiB. Since we only currently build x86(_64) bundles, adding `export XZ_OPT="--x86 --lzma2"` or similar to the top of `dtar.sh` should be sufficient, adjust presets to taste.
(There is also an Little Endian ARM BCJ filter, along with filters for architectures that we don't care about.)https://gitlab.torproject.org/legacy/trac/-/issues/13890Provide support for urdu language in Tor Browser2020-06-16T01:24:40ZArturo FilastòProvide support for urdu language in Tor BrowserCurrently Tor Browser does not support the urdu language.
Implementing this feature, though, is blocking on having an ESR of Firefox that supports such language, that will probably not happen before August 2015.
Support for urdu in Fir...Currently Tor Browser does not support the urdu language.
Implementing this feature, though, is blocking on having an ESR of Firefox that supports such language, that will probably not happen before August 2015.
Support for urdu in Firefox was just recently announced: http://mozilla-pakistan.org/2014/05/12/new-firefox-launch-event-pakistan/ and xpi's for it are only shipped in their alpha builds: https://ftp.mozilla.org/pub/mozilla.org/firefox/nightly/latest-mozilla-aurora-l10n/mac/xpi/.
When that is done the TorButton and TorLauncher components would need to be translated.
It's probably a good idea to implement this as part of #12967.https://gitlab.torproject.org/legacy/trac/-/issues/13770BusyBox-style bundling of Go programs can save space2020-06-16T01:24:40ZDavid Fifielddcf@torproject.orgBusyBox-style bundling of Go programs can save spaceMike suggested that compiling all the Go programs into one executable à la BusyBox could save space because there would be only one static copy of the Go runtime.Mike suggested that compiling all the Go programs into one executable à la BusyBox could save space because there would be only one static copy of the Go runtime.