Trac issueshttps://gitlab.torproject.org/legacy/trac/-/issues2020-06-13T17:41:17Zhttps://gitlab.torproject.org/legacy/trac/-/issues/28665tbb-testsuite: fix the fp_navigator test2020-06-13T17:41:17Zboklmtbb-testsuite: fix the fp_navigator testThe `fp_navigator` test is currently failing for non -en-US bundles with the following error:
```
AssertionError: Navigator property mismatch language [ar != en-US]
```The `fp_navigator` test is currently failing for non -en-US bundles with the following error:
```
AssertionError: Navigator property mismatch language [ar != en-US]
```https://gitlab.torproject.org/legacy/trac/-/issues/27408Make it possible to find which tor-browser-build.git commit was used to build...2020-06-16T01:25:15ZboklmMake it possible to find which tor-browser-build.git commit was used to build a nightlyCurrently it is possible to look at the nightly build logs to find which `tor-browser.git` or `tor.git` commit was used in a nightly build. However the logs don't include the `tor-browser-build.git` commit.
We should make it possible to...Currently it is possible to look at the nightly build logs to find which `tor-browser.git` or `tor.git` commit was used in a nightly build. However the logs don't include the `tor-browser-build.git` commit.
We should make it possible to find which `tor-browser-build.git` commit was used to build a nightly build.https://gitlab.torproject.org/legacy/trac/-/issues/27265In some cases, rbm will download files in the wrong project directory2020-06-13T17:39:28ZboklmIn some cases, rbm will download files in the wrong project directoryThe patch for #27045 is causing an error when starting the build with an empty `out/` directory:
https://trac.torproject.org/projects/tor/ticket/27045#comment:11
We can see in the logs that binutils is being downloaded in the `out/firef...The patch for #27045 is causing an error when starting the build with an empty `out/` directory:
https://trac.torproject.org/projects/tor/ticket/27045#comment:11
We can see in the logs that binutils is being downloaded in the `out/firefox` directory instead of `out/binutils`:
```
Saving to: '/media/ssd/Code/Tor/tor-browser-build/out/firefox/binutils-2.26.1.tar.bz2'
```
The reason is that we override `output_dir` when calling `build_pkg` in `input_files`:
```
} elsif ($input_file->{project} && $t->('project')) {
my $p = $t->('project');
print "Building project $p - $name\n";
my $run_save = $config->{run};
$config->{run} = { target => $input_file->{target} };
$config->{run}{target} //= $run_save->{target};
build_pkg($p, {%$options, origin_project => $project, %$input_file,
output_dir => $proj_out_dir});
$config->{run} = $run_save;
print "Finished build of project $p - $name\n";
} else {
```
The reason why we don't see this error in normal builds and only see it with #27045 is that in #27045 we are changing the `tor-browser` filename to remove `c("var/build_id")` from it, removing the need to download dependencies to compute the filename. In normal builds the binutils tarball is already downloaded (in the correct directory) when we start the firefox build, so we are not hitting this issue.boklmboklmhttps://gitlab.torproject.org/legacy/trac/-/issues/27136tbb-testsuite: fix dom-objects-enumeration-worker test for esr602020-06-13T17:41:13Zboklmtbb-testsuite: fix dom-objects-enumeration-worker test for esr60We need to fix the dom-objects-enumeration-worker test for the esr60 changes.
The test currently fails because the following DOM objects are now exposed in the Worker context:
```
AbortController
AbortSignal
CloseEvent
ErrorEvent
EventS...We need to fix the dom-objects-enumeration-worker test for the esr60 changes.
The test currently fails because the following DOM objects are now exposed in the Worker context:
```
AbortController
AbortSignal
CloseEvent
ErrorEvent
EventSource
FileList
PerformanceObserver
PerformanceObserverEntryList
PerformanceResourceTiming
ProgressEvent
StorageManager
WebAssembly
name
onmessageerror
origin
```boklmboklmhttps://gitlab.torproject.org/legacy/trac/-/issues/27120tbb-testsuite: marionette tests are failing when using strace2020-06-13T17:41:12Zboklmtbb-testsuite: marionette tests are failing when using straceIn the marionette tests that we run from the testsuite, we are running the browser with strace in order to check the system calls and detect possible files created outside the browser directory, and direct network connections.
When runn...In the marionette tests that we run from the testsuite, we are running the browser with strace in order to check the system calls and detect possible files created outside the browser directory, and direct network connections.
When running the testsuite on an esr60 based Tor Browser, the browser window does not open and an strace process takes a lot of cpu, until we get the following timeout error:
```
Using workspace for temporary data: "/home/user/tbb-testsuite/reports/r/FmdSZphWEU/results-tor-browser-linux64-8.0a9_es-ES.tar.xz/slider_settings_1_ws"
mozversion application_buildid: 20180204020101
mozversion application_display_name: Tor Browser
mozversion application_id: {ec8030f7-c20a-464f-9b0e-13a3a9e97384}
mozversion application_name: Firefox
mozversion application_remotingname: firefox
mozversion application_vendor: Mozilla
mozversion application_version: 60.1.0
mozversion platform_buildid: 20180204020101
mozversion platform_version: 60.1.0
Application command: /home/user/tbb-testsuite/tmp/_VycbfqfU2/tor-browser_es-ES/Browser/ff_slider_settings_1 -no-remote -marionette -profile /home/user/tbb-testsuite/reports/r/FmdSZphWEU/results-tor-browser-linux64-8.0a9_es-ES.tar.xz/slider_settings_1_ws/profile.default-1534170518
2:00.17 LOG: MainThread ERROR Failure during harness execution
Traceback (most recent call last):
File "/home/user/tbb-testsuite/virtualenv-marionette-4.3.0/local/lib/python2.7/site-packages/marionette_harness-4.3.0-py2.7.egg/marionette_harness/runtests.py", line 92, in cli
failed = harness_instance.run()
File "/home/user/tbb-testsuite/virtualenv-marionette-4.3.0/local/lib/python2.7/site-packages/marionette_harness-4.3.0-py2.7.egg/marionette_harness/runtests.py", line 72, in run
runner.run_tests(tests)
File "/home/user/tbb-testsuite/virtualenv-marionette-4.3.0/local/lib/python2.7/site-packages/marionette_harness-4.3.0-py2.7.egg/marionette_harness/runner/base.py", line 837, in run_tests
self.marionette = self.driverclass(**self._build_kwargs())
File "/home/user/tbb-testsuite/virtualenv-marionette-4.3.0/local/lib/python2.7/site-packages/marionette_driver-2.5.0-py2.7.egg/marionette_driver/marionette.py", line 621, in __init__
self.start_binary(self.startup_timeout)
File "/home/user/tbb-testsuite/virtualenv-marionette-4.3.0/local/lib/python2.7/site-packages/marionette_driver-2.5.0-py2.7.egg/marionette_driver/marionette.py", line 633, in start_binary
self.raise_for_port(timeout=timeout)
File "/home/user/tbb-testsuite/virtualenv-marionette-4.3.0/local/lib/python2.7/site-packages/marionette_driver-2.5.0-py2.7.egg/marionette_driver/marionette.py", line 710, in raise_for_port
self.host, self.port))
IOError: Process killed after 120s because no connection to Marionette server could be established. Check gecko.log for errors
```
Disabling the use of strace is fixing the issue.https://gitlab.torproject.org/legacy/trac/-/issues/27115tbb-testsuite: fix the https-everywhere-disabled test2020-06-13T17:41:11Zboklmtbb-testsuite: fix the https-everywhere-disabled testThe `https-everywhere-disabled` test is checking that some http URL is not redirected to https when HTTPS Everywhere is disabled. To disable HTTPS Everywhere we set the pref `extensions.https_everywhere.globalEnabled` to false, however t...The `https-everywhere-disabled` test is checking that some http URL is not redirected to https when HTTPS Everywhere is disabled. To disable HTTPS Everywhere we set the pref `extensions.https_everywhere.globalEnabled` to false, however this does not work anymore with the WebExtensions version.boklmboklmhttps://gitlab.torproject.org/legacy/trac/-/issues/27114tbb-testsuite: fix dom-objects-enumeration test for esr602020-06-13T17:41:11Zboklmtbb-testsuite: fix dom-objects-enumeration test for esr60We need to fix the dom-objects-enumeration test for the esr60 changes.We need to fix the dom-objects-enumeration test for the esr60 changes.boklmboklmhttps://gitlab.torproject.org/legacy/trac/-/issues/26917Update QA and Testing content on our HACKING document2020-06-16T00:48:39ZGeorg KoppenUpdate QA and Testing content on our HACKING documentOur QA and Testing content on our HACKING page needs some update.Our QA and Testing content on our HACKING page needs some update.https://gitlab.torproject.org/legacy/trac/-/issues/26149Add some ansible roles for tor browser testsuite setup2020-06-13T17:41:07ZboklmAdd some ansible roles for tor browser testsuite setupWe should add to `tor-browser-build/tools/ansible` some ansible roles to setup VMs for running the tor browser testsuite.We should add to `tor-browser-build/tools/ansible` some ansible roles to setup VMs for running the tor browser testsuite.https://gitlab.torproject.org/legacy/trac/-/issues/25849Ship tor in Tor Browser nightly builds for Windows with Rust enabled2020-06-16T00:45:39ZGeorg KoppenShip tor in Tor Browser nightly builds for Windows with Rust enabledSimilar to #25481 we want to start shipping tor with Rust support enabled in our nightly builds for Windows.Similar to #25481 we want to start shipping tor with Rust support enabled in our nightly builds for Windows.https://gitlab.torproject.org/legacy/trac/-/issues/25835Move build dependencies from people.tpo2020-06-16T01:25:06ZboklmMove build dependencies from people.tpoCurrently we are hosting some of the dependencies we use for building Tor Browser on people.torproject.org. However people.torproject.org is just one host, and it is sometimes down.
I think we could maybe move those files to extra.torpr...Currently we are hosting some of the dependencies we use for building Tor Browser on people.torproject.org. However people.torproject.org is just one host, and it is sometimes down.
I think we could maybe move those files to extra.torproject.org, dist.torproject.org, or ask tpa to create a new hostname for those files.https://gitlab.torproject.org/legacy/trac/-/issues/25779Ship tor in Tor Browser nightly builds for macOS with Rust enabled2020-06-16T00:45:30ZGeorg KoppenShip tor in Tor Browser nightly builds for macOS with Rust enabledSimilar to #25481 we want to start shipping tor with Rust support enabled in our nightly builds for macOS.Similar to #25481 we want to start shipping tor with Rust support enabled in our nightly builds for macOS.https://gitlab.torproject.org/legacy/trac/-/issues/25421Add an option to clean repository if checkout fails2020-06-13T17:39:24ZboklmAdd an option to clean repository if checkout failsOur nightly build sometimes fail with an error such as:
```
Error: Cannot checkout tor-browser-52.6.0esr-8.0-2-build2
```
The reason is that the working tree from the git repository is not clean, which prevents checking out an other com...Our nightly build sometimes fail with an error such as:
```
Error: Cannot checkout tor-browser-52.6.0esr-8.0-2-build2
```
The reason is that the working tree from the git repository is not clean, which prevents checking out an other commit.
We should add an option to rbm to do a `git reset --hard` and `git clean` if the checkout fails.https://gitlab.torproject.org/legacy/trac/-/issues/24332Add a script to convert code signed OSX bundles from tar to dmg2020-06-16T01:25:03ZboklmAdd a script to convert code signed OSX bundles from tar to dmgDuring the signing process we code-sign the OSX bundles on our signing machine, creating tarballs containing the OSX bundles. After this step is done we need to convert those tarballs to dmg files using some build machine. We can add a s...During the signing process we code-sign the OSX bundles on our signing machine, creating tarballs containing the OSX bundles. After this step is done we need to convert those tarballs to dmg files using some build machine. We can add a script to `tor-browser-build` doing that using the same tools we use to generate the unsigned dmg files.https://gitlab.torproject.org/legacy/trac/-/issues/23657Decide directories used for signed/unsigned builds in tor-browser-builds2020-06-16T01:25:00ZboklmDecide directories used for signed/unsigned builds in tor-browser-buildsWith gitian builds, all builds (signed and unsigned) were stored in the gitian directory directly.
I remember having issues when generating incremental mars which were done with unsigned osx mars instead of the signed ones. To avoid tha...With gitian builds, all builds (signed and unsigned) were stored in the gitian directory directly.
I remember having issues when generating incremental mars which were done with unsigned osx mars instead of the signed ones. To avoid that issue I have started with separating signed and unsigned builds in tor-browser-build. However it seems the new process is confusing and error-prone during the signing process, so we should think more about what directories we want to use in the different steps (or if we want to go back to using only one).
I think having different directories could also be useful if we want to add some scripts helping with the intermediate signing steps.
What we currently have is:
- the build target creates builds in the directory `unsigned`
- the incrementals target generates incremental mars in the `unsigned` directory. However it is using/downloading the mar files from the old version in the `unsigned` directory too, while it should be done from a signed build (for the osx ones). Maybe we should fix the incrementals step to use the old version from the `signed` directory instead.
- the dmg2mar target is using the `signed` directory. However at this point, only the dmg files are signed, so it is confusing to put all the files in the `signed` directory. Maybe an intermediate directory should be used instead?
- the update_responses target is using the `signed` directory. I think this one is correct as update responses should only be done from a fully signed build.https://gitlab.torproject.org/legacy/trac/-/issues/23386Run Tor Browser testsuite on our Windows QA machine2020-06-13T17:41:03ZboklmRun Tor Browser testsuite on our Windows QA machineWith #23315 we now have a new VM for Windows QA. We need to set up the testuite on this machine to run it on new releases and nightly builds.With #23315 we now have a new VM for Windows QA. We need to set up the testuite on this machine to run it on new releases and nightly builds.https://gitlab.torproject.org/legacy/trac/-/issues/21404Run the Tor Browser testsuite on rbm based nightly builds2020-06-13T17:40:56ZboklmRun the Tor Browser testsuite on rbm based nightly buildsIn #21286 some nightly builds have been added. We should run the testsuite on those nightly builds.In #21286 some nightly builds have been added. We should run the testsuite on those nightly builds.https://gitlab.torproject.org/legacy/trac/-/issues/12968Specify HEASLR (High Entropy Address Space Layout Randomization) in MinGW-w642020-06-16T01:24:37ZMike PerrySpecify HEASLR (High Entropy Address Space Layout Randomization) in MinGW-w64Mozilla patched mingw-w64 to allow the specification of "high-entropy" ASLR, which is an extra hardened ASLR option on Windows. Not sure if this flag only applies to 64bit builds. I think it might.
Here's their ticket:
https://github.co...Mozilla patched mingw-w64 to allow the specification of "high-entropy" ASLR, which is an extra hardened ASLR option on Windows. Not sure if this flag only applies to 64bit builds. I think it might.
Here's their ticket:
https://github.com/rust-lang/rust/issues/16593
and the patch:
https://sourceware.org/ml/binutils/2014-08/msg00167.html
and it's approval by the Binutils team:
https://sourceware.org/ml/binutils/2014-08/msg00177.html