tor-browser-bundle-testsuite issueshttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues2023-12-08T15:19:37Zhttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40078Update tools/tb-build-06-start-nightly-build after tor-browser-build#408292023-12-08T15:19:37ZboklmUpdate tools/tb-build-06-start-nightly-build after tor-browser-build#40829In tor-browser-build#40829 we changed the name of browser
installer/package artifacts, but `tools/tb-build-06-start-nightly-build`
is still using the old names (when copying files for archiving).In tor-browser-build#40829 we changed the name of browser
installer/package artifacts, but `tools/tb-build-06-start-nightly-build`
is still using the old names (when copying files for archiving).https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40077Create .nobackup file in tor-browser-build directory when starting nightly build2023-10-21T17:55:02ZboklmCreate .nobackup file in tor-browser-build directory when starting nightly buildWhen starting a nightly build, we should make sure a `.nobackup` file
exists in the `tor-browser-build` directory, to exclude it from backups.When starting a nightly build, we should make sure a `.nobackup` file
exists in the `tor-browser-build` directory, to exclude it from backups.boklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40070Check updater exception for readelf_no_runpath test2023-01-05T13:01:14ZAlex CatarineuCheck updater exception for readelf_no_runpath testIn 78 we had to add `updater` to the `skip_files` for the `readelf_no_runpath` test. We should investigate this and possibly try to remove it from the skip list.In 78 we had to add `updater` to the `skip_files` for the `readelf_no_runpath` test. We should investigate this and possibly try to remove it from the skip list.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40069Do cleaning of build artifacts less often in nightly builds2023-08-26T06:06:08ZboklmDo cleaning of build artifacts less often in nightly buildsWe currently run `make clean` each day during nightly builds, which takes a lot of time. Since multi-locale bundles take less space, and we have some space available on `tb-build-01`, I think we could do cleaning less often. Maybe every ...We currently run `make clean` each day during nightly builds, which takes a lot of time. Since multi-locale bundles take less space, and we have some space available on `tb-build-01`, I think we could do cleaning less often. Maybe every 5 builds?Sponsor 131 - Phase 4 - Browser Release Managementboklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40066Make tools/signing/nightly/sign-nightly sign mar files when there is no incre...2022-10-21T18:18:00ZboklmMake tools/signing/nightly/sign-nightly sign mar files when there is no incrementals`tools/signing/nightly/sign-nightly` currently check for the file `sha256sums-unsigned-build.incrementals.txt` to decide if a new version is ready to be signed. This means that when building incrementals failed, the mar files are not sig...`tools/signing/nightly/sign-nightly` currently check for the file `sha256sums-unsigned-build.incrementals.txt` to decide if a new version is ready to be signed. This means that when building incrementals failed, the mar files are not signed.Sponsor 131 - Phase 4 - Browser Release Managementboklmboklmhttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40060Write test for spoofed navigator* values in workers2022-07-09T21:51:25ZGeorg KoppenWrite test for spoofed navigator* values in workerslegacy/trac#13027 fixed the problem with navigator* values not being spoofed. Back then we could not write a Mozmill test as Mozmill itself was affected by the underlying issue (https://bugzilla.mozilla.org/show_bug.cgi?id=1062920 togeth...legacy/trac#13027 fixed the problem with navigator* values not being spoofed. Back then we could not write a Mozmill test as Mozmill itself was affected by the underlying issue (https://bugzilla.mozilla.org/show_bug.cgi?id=1062920 together with https://bugzilla.mozilla.org/show_bug.cgi?id=1078163).https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40059Investigate tbb-test for #23104 (line height compensation)2022-12-08T15:15:25ZAlex CatarineuInvestigate tbb-test for #23104 (line height compensation)After the changes from tor-browser#30832 we set `layout.css.line-height.normal-as-resolved-value.enabled = false` in order to measure the `line-height` correctly for the arabic test case (see https://gitlab.torproject.org/tpo/application...After the changes from tor-browser#30832 we set `layout.css.line-height.normal-as-resolved-value.enabled = false` in order to measure the `line-height` correctly for the arabic test case (see https://gitlab.torproject.org/tpo/applications/tor-browser/-/issues/30832#note_2664115). We should try to investigate and possibly fix the test case so that we don't have to set that pref.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40058Add support for Fenix into existing Tor Browser regression and integration tests2022-07-08T23:09:57ZMatthew FinkelAdd support for Fenix into existing Tor Browser regression and integration testsIntegrate Fenix into test suiteIntegrate Fenix into test suitehttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40057add automated test to ensure OS-provided certificates are not used2022-07-08T21:22:02ZMark Smithadd automated test to ensure OS-provided certificates are not usedFrom tor-browser#33534: Firefox 75 can use OS-provided certificates. This fingerprinting vector is disabled by default but we should add an automated test to confirm that it stays off during future rebasings.
https://blog.mozilla.org/se...From tor-browser#33534: Firefox 75 can use OS-provided certificates. This fingerprinting vector is disabled by default but we should add an automated test to confirm that it stays off during future rebasings.
https://blog.mozilla.org/security/2020/04/14/expanding-client-certificates-in-firefox-75/https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40056mochitest failures for #23104 patch2022-06-23T19:11:59ZArthur Edelsteinmochitest failures for #23104 patchI ran the mochitests for the legacy/trac#23104 patch on our tor-browser-52.6.0esr-8.0-2 branch on Debian GNU/Linux 9.1 and I ran into the following failures:
```
Passed: 3
Failed: 3
Todo: 0
failed | Line Height validation - got 20.4, ex...I ran the mochitests for the legacy/trac#23104 patch on our tor-browser-52.6.0esr-8.0-2 branch on Debian GNU/Linux 9.1 and I ran into the following failures:
```
Passed: 3
Failed: 3
Todo: 0
failed | Line Height validation - got 20.4, expected 19.8
failed | Line Height validation - got 20.4, expected 19.8
failed | Line Height validation - got 20.4, expected 19.8
passed | Line Height validation
passed | Line Height validation
passed | Line Height validation
```https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40055Unit test for js locale2022-08-04T21:45:08ZArthur EdelsteinUnit test for js localeLet's introduce an automated regression test to ensure that the `javascript.use_us_english_locale` pref is applied to
* members of `window.Date` objects
* Intl API
* toLocaleString
* `DateTimeFormat.formatToParts`
I can work on this at ...Let's introduce an automated regression test to ensure that the `javascript.use_us_english_locale` pref is applied to
* members of `window.Date` objects
* Intl API
* toLocaleString
* `DateTimeFormat.formatToParts`
I can work on this at some point. The problem is that we need to set the LANG variable to de_DE or another non-en_US locale before starting Firefox. Maybe boklm's testing framework can do this? Or maybe it's possible in Mozilla's automated testing system, but I don't know a way yet.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40054Regression tests for canvas image extraction prompt2022-08-04T21:45:09ZArthur EdelsteinRegression tests for canvas image extraction promptIt would be nice to have regression tests for our Canvas Image Extraction patch (legacy/trac#6253).
mcs and brade developed some tests here: https://people.torproject.org/~brade/tests/canvasTest.html. Perhaps we could port these to a mo...It would be nice to have regression tests for our Canvas Image Extraction patch (legacy/trac#6253).
mcs and brade developed some tests here: https://people.torproject.org/~brade/tests/canvasTest.html. Perhaps we could port these to a mochitest.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40053Test cases for OCSP isolation2022-08-04T21:45:10ZArthur EdelsteinTest cases for OCSP isolationWe need some sort of regression tests for OCSP isolation, which was implemented in legacy/trac#13670.2.We need some sort of regression tests for OCSP isolation, which was implemented in legacy/trac#13670.2.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40052Unit tests for font whitelisting patch2022-06-17T20:13:46ZArthur EdelsteinUnit tests for font whitelisting patchWe need unit tests for the font whitelisting patch (legacy/trac#13313).We need unit tests for the font whitelisting patch (legacy/trac#13313).https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40051Regression tests for Bug #2950 (Make Permissions Manager memory-only) never f...2022-06-16T03:55:20ZboklmRegression tests for Bug #2950 (Make Permissions Manager memory-only) never failsThe regression test for bug legacy/trac#2950 does not fail when used on 38.2.0esr without patch:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a81cdc18350bThe regression test for bug legacy/trac#2950 does not fail when used on 38.2.0esr without patch:
https://treeherder.mozilla.org/#/jobs?repo=try&revision=a81cdc18350bhttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40050Verify that Tor Browser patch reverts cause test failures2022-10-05T23:35:32ZArthur EdelsteinVerify that Tor Browser patch reverts cause test failuresWe should verify that if we revert any of our Tor Browser patches that have a corresponding regression test, then the test will fail.
We have regression tests for the following patches:
* Bug legacy/trac#5856: Do not expose physical scr...We should verify that if we revert any of our Tor Browser patches that have a corresponding regression test, then the test will fail.
We have regression tests for the following patches:
* Bug legacy/trac#5856: Do not expose physical screen info via window & window.screen.
* Bug legacy/trac#2875: Limit device and system specific CSS Media Queries.
* Bug legacy/trac#2950: Make Permissions Manager memory-only
* Bug legacy/trac#4755: Return client window coordinates for mouse event screenX/Y (for dragend, 0,0 is returned).
* Bug legacy/trac#2874: Block Components.interfaces from content
Additionally the tests in
* Bug legacy/trac#13749.1: regression tests for first party isolation of localStorage
* Bug legacy/trac#13749.2: Regression tests for first-party isolation of cache
cover patches
* Bug legacy/trac#6564: Isolate DOM storage to first party URI.
* Bug legacy/trac#6539: Isolate the Image Cache per url bar domain.
* Bug legacy/trac#13742: Isolate cache to URL bar domain.
* Bug legacy/trac#10819: Add a pref, "privacy.thirdparty.isolate", to allow the activation or deactivation of isolating DOM storage and image caching by first party URI.
There are also a couple of regression test patches that I think may be broken:
* TB4: Tor Browser's Firefox preference overrides.
* Omnibox: Add DDG, Startpage, Disconnect, Youtube, Twitter; remove Amazon, eBay, bingSponsor 131 - Phase 5 - Ongoing Maintenancehttps://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40049Unit tests to confirm content pref service doesn't write to disk in PBM2022-06-16T00:21:49ZArthur EdelsteinUnit tests to confirm content pref service doesn't write to disk in PBMIf we can confirm that the Content Pref Service doesn't write to disk in private browsing mode, then we can drop our patch for legacy/trac#3229. See the discussion at https://bugzilla.mozilla.org/show_bug.cgi?id=967809
Probably the most...If we can confirm that the Content Pref Service doesn't write to disk in private browsing mode, then we can drop our patch for legacy/trac#3229. See the discussion at https://bugzilla.mozilla.org/show_bug.cgi?id=967809
Probably the most secure way to do this is to write unit tests.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40048add automated tests for svg.in-content.enabled2022-06-16T00:14:53ZMark Smithadd automated tests for svg.in-content.enabledAs a followup to legacy/trac#12827 and legacy/trac#15794, we should (as Mike suggested) add a bunch of tests for various SVG-related scenarios including things such as use of <style> and <img> tags within <svg> blocks. Kathy and I teste...As a followup to legacy/trac#12827 and legacy/trac#15794, we should (as Mike suggested) add a bunch of tests for various SVG-related scenarios including things such as use of <style> and <img> tags within <svg> blocks. Kathy and I tested quite a few scenarios when we did the work for legacy/trac#12827 but obviously we missed at least one.
I am opening this new, separate ticket so we do not lose track of this work item.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40047Regression tests for #5926 patch2022-06-15T02:45:11ZArthur EdelsteinRegression tests for #5926 patchThis regression test should cover all problems reported in legacy/trac#5926, legacy/trac#10284, and legacy/trac#13019.This regression test should cover all problems reported in legacy/trac#5926, legacy/trac#10284, and legacy/trac#13019.https://gitlab.torproject.org/tpo/applications/tor-browser-bundle-testsuite/-/issues/40044Create a test for proxy usage in Moat requests2022-08-04T21:45:05ZPier Angelo VendrameCreate a test for proxy usage in Moat requestsIn addition to checking that Tor works over proxies (#40043) we should add tests that:
1. Moat requests can be done through a proxy as well (this can make finding the location harder for S96-related work, but it may be compulsory for so...In addition to checking that Tor works over proxies (#40043) we should add tests that:
1. Moat requests can be done through a proxy as well (this can make finding the location harder for S96-related work, but it may be compulsory for some networks)
2. the proxy settings that are set in the advanced preferences panel are used also for this