Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-16T01:25:25Zhttps://gitlab.torproject.org/legacy/trac/-/issues/29041Compile clang closer to how Mozilla does it2020-06-16T01:25:25ZGeorg KoppenCompile clang closer to how Mozilla does itWe compile clang differently to how Mozilla does it: Mozilla has an elaborate setup with three stages (if needed) (see: https://hg.mozilla.org/releases/mozilla-esr60/file/248ca5c585f8/build/build-clang/build-clang.py) while we essentiall...We compile clang differently to how Mozilla does it: Mozilla has an elaborate setup with three stages (if needed) (see: https://hg.mozilla.org/releases/mozilla-esr60/file/248ca5c585f8/build/build-clang/build-clang.py) while we essentially "just" do what is outlined on https://clang.llvm.org/get_started.html.
We should change that and get our toolchain closer to what Mozilla provides.https://gitlab.torproject.org/legacy/trac/-/issues/28830Clean up tor-browser-build build scripts/config files2020-06-16T01:25:25ZGeorg KoppenClean up tor-browser-build build scripts/config filesWe should go over the projects we have in `tor-browser-build` and clean up build scripts and config files if needed.
Areas we could/should cover:
1) We sometimes have several platform related blocks in one build script which could be c...We should go over the projects we have in `tor-browser-build` and clean up build scripts and config files if needed.
Areas we could/should cover:
1) We sometimes have several platform related blocks in one build script which could be confusing. Can we merge those (or some of them) while still keeping the overall flow of the script logic?
2) Duplicating platform-specific and !platform-specific commands, e.g. `cd $distdir` in
```
[% IF c("var/osx") %]
LIBEVENT_FILE=`basename $libeventdir/lib/libevent-*.dylib`
cd $distdir
cp bin/tor $TORBINDIR/
cd $TORBINDIR/
x86_64-apple-darwin11-install_name_tool -change $libeventdir/lib/$LIBEVENT_FILE @executable_path/$LIBEVENT_FILE tor
[% END %]
cd $distdir
```
3)...https://gitlab.torproject.org/legacy/trac/-/issues/28782Use Path::Tiny instead of File::Slurp in tools/clean-old and tools/dmg2mar2020-06-16T01:25:24ZboklmUse Path::Tiny instead of File::Slurp in tools/clean-old and tools/dmg2marSimilarly to #24361 and #28771, we should replace uses of File::Slurp by Path::Tiny in tools/clean-old and tools/dmg2mar.Similarly to #24361 and #28771, we should replace uses of File::Slurp by Path::Tiny in tools/clean-old and tools/dmg2mar.https://gitlab.torproject.org/legacy/trac/-/issues/28754make testbuild-android-armv7 stalls during sha256sum step2020-06-16T01:25:23ZGeorg Koppenmake testbuild-android-armv7 stalls during sha256sum stepSuppose you don't have a testbuild directory containing any already built bundles. Now, start a build with `make testbuild-android-armv7` to get an .apk. The result is a stalled build in the release project.
The reason for that is that
...Suppose you don't have a testbuild directory containing any already built bundles. Now, start a build with `make testbuild-android-armv7` to get an .apk. The result is a stalled build in the release project.
The reason for that is that
```
sha256sum $(ls -1 *.exe *.tar.xz *.dmg *.mar *.zip *.tar.gz | grep -v '\.incremental\.mar$' | sort) > sha256sums-unsigned-build.txt
```
evaluates to
```
sha256sum > sha256sums-unsigned-build.txt
```
which waits for input. We have not been hitting that so far as both in the alpha and the nightly case other bundles in the directory are available. And for the mobile case #25164 is about to fix this as well. However, we might want to be a bit more conservative and only execute `sha256sum` if we actually have an argument to pass. Otherwise this can lead to surprising results (like in this case).https://gitlab.torproject.org/legacy/trac/-/issues/28595Remove the need to update var/gradle_dependencies_version2020-06-16T01:25:23ZboklmRemove the need to update var/gradle_dependencies_versionCurrently we need to update `var/gradle_dependencies_version` each time `gradle-dependencies-list.txt` is updated.
If you change `gradle-dependencies-list.txt` without updating `var/gradle_dependencies_version`, then you can easily end ...Currently we need to update `var/gradle_dependencies_version` each time `gradle-dependencies-list.txt` is updated.
If you change `gradle-dependencies-list.txt` without updating `var/gradle_dependencies_version`, then you can easily end up with the wrong list of gradle dependencies being used in a build.
To avoid this we should see if we can make the gradle dependencies be fetched again automatically when `gradle-dependencies-list.txt` has been updated, without the need to update `var/gradle_dependencies_version`.https://gitlab.torproject.org/legacy/trac/-/issues/28326Tor Browser for PPC64LE2020-06-16T01:25:22ZTracTor Browser for PPC64LEI'm here to request a Tor Browser for Debian to get compiled for ppc64le architecture.
Power architecture is the only open hardware, so i think it's important for a project like Tor to support this architecture, avoiding potential backdo...I'm here to request a Tor Browser for Debian to get compiled for ppc64le architecture.
Power architecture is the only open hardware, so i think it's important for a project like Tor to support this architecture, avoiding potential backdoor on closed system.
The follow link could probably be usefull for build it correctly
https://www.talospace.com/2018/09/more-power-in-firefox-62.html
https://www.talospace.com/2018/10/patches-needed-for-firefox-63.html
If you need a ppc64le machine it will provided a cloud access for free, ask to https://twitter.com/RaptorCompSys for building and testing Tor Browser
No secure comunications is really secure on closed system, so to support an open architecture should be a priority for a project who looking for a digital freedom
**Trac**:
**Username**: power9https://gitlab.torproject.org/legacy/trac/-/issues/28325Use go 1.11 module versioning support2020-06-16T01:25:22ZboklmUse go 1.11 module versioning supportGo 1.11 is adding preliminary support for Go modules, replacing the GOPATH-based approach.
obfs4 is adding `go.mod` and `go.sum` files, which use the Go modules support to fetch the dependencies. We disable that in #28310 as the go libr...Go 1.11 is adding preliminary support for Go modules, replacing the GOPATH-based approach.
obfs4 is adding `go.mod` and `go.sum` files, which use the Go modules support to fetch the dependencies. We disable that in #28310 as the go libraries we build are not currently recognized as Go modules.
We should change how we build our Go libraries so that they can be used as Go modules.https://gitlab.torproject.org/legacy/trac/-/issues/28240Think about using mingw-w64 with dwarf exception support for rustc cross-comp...2020-06-16T01:25:20ZGeorg KoppenThink about using mingw-w64 with dwarf exception support for rustc cross-compilation for 32bit WindowsOur workaround we currently use to cope with issues cross-compiling rustc for 32bit Windows is not upstreamable (see: https://github.com/rust-lang/rust/pull/55444 for the arguments). We could think about compiling with a mingw-w64 using ...Our workaround we currently use to cope with issues cross-compiling rustc for 32bit Windows is not upstreamable (see: https://github.com/rust-lang/rust/pull/55444 for the arguments). We could think about compiling with a mingw-w64 using dwarf exception support instead, see: (https://github.com/rust-embedded/cross/blob/master/docker/mingw.sh).
However, we should take into account that we want to switch to a mingw-w64/clang-based toolchain soon and should start looking into this ticket not before the other (#28238) is done.https://gitlab.torproject.org/legacy/trac/-/issues/28176Cleanup and add the testsuite to tor-browser-build.git2020-06-16T01:25:20ZboklmCleanup and add the testsuite to tor-browser-build.gitCurrently we have a Tor Browser testsuite in a separate git repository, at https://gitweb.torproject.org/user/boklm/tor-browser-bundle-testsuite.git/.
Rather than using a separate repository, I think it might be better to integrate that...Currently we have a Tor Browser testsuite in a separate git repository, at https://gitweb.torproject.org/user/boklm/tor-browser-bundle-testsuite.git/.
Rather than using a separate repository, I think it might be better to integrate that testsuite directly into `tor-browser-build.git`, as a sub-directory:
- quite often some Tor Browser changes need corresponding changes in the testsuite. Having both in the same repository would make it possible to do both changes with the same commit (although we might still need two commits when the change is in `tor-browser.git`), and use the same review process.
- some people don't know that we have a testsuite, or where it is. Having it in the same repository would make it easier to find and know that it exists.
- having the testsuite in the same repository would make it easier to integrate it with the build process. For example we could add some makefile command such as `make run-testsuite-testbuild`, which would make it easier for developers to run the testsuite after doing a build.
One option would be to copy the current testsuite to some `tor-browser-build.git` sub-directory, and make necessary changes to fix it and integrate it better. However I think that over time the testsuite accumulated a lot of complexity that is not really needed. Adding the current testsuite and fixing it will probably result in adding even more complexity. I think that an other option is to do some important cleanup first and only import minimal testsuite tools, and then cleanup and import each of the tests one by one.
So I think the main two options are:
1) copy the current testsuite, fix all the tests, then try to integrate it better, and to clean it.
2) rewrite minimal testsuite tools, avoiding the complexity from the previous version, and import each test one by one after fixing and cleaning them.
To me it seems easier to re-start from a minimal testsuite and re-add the needed features and tests (sometimes copying them from the old testsuite, and sometimes re-writting them from scratch if it makes more sense) than taking the current testsuite and trying to clean it little by little, so I think option 2) is better.https://gitlab.torproject.org/legacy/trac/-/issues/28175Add commands to upload bundles to virustotal2020-06-16T01:25:19ZboklmAdd commands to upload bundles to virustotalWith each new release, some users are complaining that their Windows antivirus is detecting Tor Browser as a virus. A theory is that the antivirus is blocking programs that they have not seen before until they are a few days old.
To avo...With each new release, some users are complaining that their Windows antivirus is detecting Tor Browser as a virus. A theory is that the antivirus is blocking programs that they have not seen before until they are a few days old.
To avoid this problem (or at least try to limit the duration of the problem), we could upload the Windows bundles to virustotal.com as soon as we have a build ready, so that the antivirus have seen them a few days/hours before the release.
In the past we had something as part of the testsuite uploading the bundles to virustotal, but it is currently broken and needs to be fixed. I think we could add some makefile commands to `tor-browser-build.git` such as `make upload-unsigned-build-virustotal-release` (with `*-signed-*` and `*-alpha` variants) to upload a build to virustotal (in addition to having it done by the testsuite, probably using the same script for both).https://gitlab.torproject.org/legacy/trac/-/issues/28156Start building Tor with NSS support in Tor Browser2020-06-16T01:25:19ZGeorg KoppenStart building Tor with NSS support in Tor BrowserWe should try to get rid of OpenSSL if we can and compile Tor to use NSS which we ship anyway.We should try to get rid of OpenSSL if we can and compile Tor to use NSS which we ship anyway.https://gitlab.torproject.org/legacy/trac/-/issues/28124Show Tor Browser icon as macOS volume icon2020-06-16T01:25:18ZGeorg KoppenShow Tor Browser icon as macOS volume iconIIRC we show the generic volume icon on macOS instead of the Tor Browser one. We should try to show ours instead, though. It seems Mozills did this within their cross-compilation setup a while back (https://bugzilla.mozilla.org/show_bug....IIRC we show the generic volume icon on macOS instead of the Tor Browser one. We should try to show ours instead, though. It seems Mozills did this within their cross-compilation setup a while back (https://bugzilla.mozilla.org/show_bug.cgi?id=1197325) and we might benefit fromt that work.https://gitlab.torproject.org/legacy/trac/-/issues/28102Make sure we pick the exact same compile environment for Tor Browser builds2020-06-16T01:25:18ZGeorg KoppenMake sure we pick the exact same compile environment for Tor Browser buildsWhen creating a container image at time t and creating a container image at time t1 it is not guaranteed that we include the same compiler versions etc.
This has led to `mar` executables being different, for instance (see: comment:52:ti...When creating a container image at time t and creating a container image at time t1 it is not guaranteed that we include the same compiler versions etc.
This has led to `mar` executables being different, for instance (see: comment:52:ticket:26475): GCC version was in both cases 4.9.2 but in one of them contained an updated resulting in `GCC: (Debian 4.9.2-10+deb8u1) 4.9.2` as a version string instead of `GCC: (Debian 4.9.2-10) 4.9.2`.
We could fix that by making sure we compile our own GCC for this case but there might be other packages lurking with the same trouble potential. We should make sure instead that everyone creating *and* using a container image is getting/having the same available.https://gitlab.torproject.org/legacy/trac/-/issues/27551Build HTTPS Everywhere with the SecureDrop custom ruleset update channel2020-06-16T01:25:16ZWilliam BudingtonBuild HTTPS Everywhere with the SecureDrop custom ruleset update channelRelated: https://github.com/freedomofpress/securedrop/issues/3668, https://github.com/EFForg/https-everywhere/blob/master/docs/en_US/ruleset-update-channels.md, https://blog.torproject.org/cooking-onions-names-your-onions
Blocked by: ht...Related: https://github.com/freedomofpress/securedrop/issues/3668, https://github.com/EFForg/https-everywhere/blob/master/docs/en_US/ruleset-update-channels.md, https://blog.torproject.org/cooking-onions-names-your-onions
Blocked by: https://trac.torproject.org/projects/tor/ticket/10394
There has been some discussion on the SecureDrop issue tracker on the possibility of FPF maintaining their own SecureDrop custom rulesets update channel.
A "ruleset" is a strictly formatted XML file that defines how HTTPS Everywhere issues redirects. HTTPS Everywhere comes bundled with thousands of these rulesets that tell it how to secure sites. An "update channel" is a new way for HTTPS Everywhere to have rulesets delivered to the extension without publishing a new version of the extension itself. Rulesets are periodically checked for on a publicly accessible, known URL and verified via the WebCrypto API, see https://github.com/EFForg/https-everywhere/blob/master/docs/en_US/ruleset-update-channels.md#update-channel-format--logic for more info on how this works. Currently the Tor Browser includes the `EFF (Full)` update channel. Requests to the update channels to check for ruleset updates are made every 24 hours.
The idea here is for FPF to maintain its own update channel which forwards URLs of the format `http://0.0.0.0/theintercept.com.securedrop` to the onion URL for that SecureDrop instance. This gives a few distinct advantages:
1. It allows easier discovery of SecureDrop sites
2. It allows FPF to quickly rotate keys in case of key compromise or vulnerabilities, as has happened in the past with HeartBleed (https://freedom.press/news/securedrop-and-the-openssl-vulnerability/)
3. If someone accesses a site of this format in Chrome browser, it will not leak the DNS request. This provides no additional security in Firefox, since Firefox already blocks DNS requests for the `.onion` pseudo-TLD.
Understandably, Tor Browser may be wary of adding an additional update channel, maintained by another entity, which has the arbitrary ability to redirect URLs. That's why HTTPS Everywhere has added the concept of `scope` to update channels. `scope` defines some regular expression for URLs on which a ruleset update channel is allowed to operate. For instance, in the case of SecureDrop, you can define the scope to be `https?://0\.0\.0\.0/[^/]+\.securedrop($|/)` so that this update channel only operate on URLs of the format given above.
This update channel would have to be added to HTTPS Everywhere for the Tor Browser (in https://github.com/EFForg/https-everywhere/blob/master/chromium/background-scripts/update_channels.js) at build-time. Since all customizations are overwritten when the extension is updated, this is also blocked by https://trac.torproject.org/projects/tor/ticket/10394https://gitlab.torproject.org/legacy/trac/-/issues/27457Add architecture to 32bit Windows bundle filename2020-06-16T01:25:16ZGeorg KoppenAdd architecture to 32bit Windows bundle filenameThe 32bit Windows bundles are the only ones where the architecture (i.e. "win32") is missing in the filename. I think that's more or less for historical reasons and we should change that.The 32bit Windows bundles are the only ones where the architecture (i.e. "win32") is missing in the filename. I think that's more or less for historical reasons and we should change that.https://gitlab.torproject.org/legacy/trac/-/issues/27408Make it possible to find which tor-browser-build.git commit was used to build...2020-06-16T01:25:15ZboklmMake it possible to find which tor-browser-build.git commit was used to build a nightlyCurrently it is possible to look at the nightly build logs to find which `tor-browser.git` or `tor.git` commit was used in a nightly build. However the logs don't include the `tor-browser-build.git` commit.
We should make it possible to...Currently it is possible to look at the nightly build logs to find which `tor-browser.git` or `tor.git` commit was used in a nightly build. However the logs don't include the `tor-browser-build.git` commit.
We should make it possible to find which `tor-browser-build.git` commit was used to build a nightly build.https://gitlab.torproject.org/legacy/trac/-/issues/27045Add option for firefox incremental builds2020-06-16T01:25:15ZboklmAdd option for firefox incremental buildsIn Tor Browser builds, when we need to rebuild firefox, we do a full firefox rebuild. During the development of firefox patches it would be useful to be able to do incremental rebuilds of firefox.
To do that, we could add a new makefile...In Tor Browser builds, when we need to rebuild firefox, we do a full firefox rebuild. During the development of firefox patches it would be useful to be able to do incremental rebuilds of firefox.
To do that, we could add a new makefile target for this type of build and an option in `rbm.local.conf` pointing to the firefox sources directory. We would then mount that directory in the build container and use it instead of the sources from git.https://gitlab.torproject.org/legacy/trac/-/issues/26895'Error downloading build result' after GCC in container Tor Browser Bundle Bu...2020-06-16T01:25:13ZTrac'Error downloading build result' after GCC in container Tor Browser Bundle Build (rbm)... most descriptive error ever ocurred in the world $ make
git submodule update --init
./rbm/rbm build release --target release --target torbrowser-all
Building project tor-browser - tor-browser-8.0a9-linux-x86_64-58eca3
Using file out/container-image/container-image_w... $ make
git submodule update --init
./rbm/rbm build release --target release --target torbrowser-all
Building project tor-browser - tor-browser-8.0a9-linux-x86_64-58eca3
Using file out/container-image/container-image_wheezy-amd64-df3a332e7b34.tar.gz
Building project firefox - firefox-a0efd2fcd6e9-linux-x86_64-0974b5
Tag tor-browser-60.1.0esr-8.0-1-build2 is signed with key 35CD74C24A9B15A19E1A81A194373AA94B7C3223
Created tmp/rbm-mgl5M/firefox-a0efd2fcd6e9.tar.gz
Using file out/container-image/container-image_wheezy-amd64-279bd3a261cd.tar.gz
Building project gcc - gcc-6.4.0-3098e6.tar.gz
Using file out/container-image/container-image_wheezy-amd64-2bf0a6561acb.tar.gz
Using file out/gcc/gcc-6.4.0.tar.xz
[sudo] password for $USER:
Build log: logs/gcc-linux-x86_64.log
[sudo] password for $USER:
Error: Error downloading build result
make: *** [Makefile:6: release] Error 1
Thats the log on the screen...
**Trac**:
**Username**: kfseapersonhttps://gitlab.torproject.org/legacy/trac/-/issues/26877Declare gcc version in rbm.conf2020-06-16T01:25:12ZSukhbir SinghDeclare gcc version in rbm.confWe declare the `gcc` version (currently set to `6.4.0`) in multiple places:
```
projects/gcc/config:version: 6.4.0
```
```
projects/mingw-w64/config: gcc_version: 6.4.0
```
We should probably define it in `rbm.conf` instead and then ...We declare the `gcc` version (currently set to `6.4.0`) in multiple places:
```
projects/gcc/config:version: 6.4.0
```
```
projects/mingw-w64/config: gcc_version: 6.4.0
```
We should probably define it in `rbm.conf` instead and then refer to that. (This may also be relevant for #25485).https://gitlab.torproject.org/legacy/trac/-/issues/26417Tor Browser build is not working with the runc version in debian testing2020-06-16T01:25:11ZboklmTor Browser build is not working with the runc version in debian testingahf reported that building tor browser on Debian testing fails with the following error:
```
Error: Error starting remote:
json: cannot unmarshal array into Go struct field Process.capabilities of type specs.LinuxCapabilities
Makefile:...ahf reported that building tor browser on Debian testing fails with the following error:
```
Error: Error starting remote:
json: cannot unmarshal array into Go struct field Process.capabilities of type specs.LinuxCapabilities
Makefile:66: recipe for target 'nightly-osx-x86_64' failed
```
The output of `runc --version` on Debian testing is:
```
$ /usr/sbin/runc --version
runc version spec: 1.0.1
```