1. 04 Jun, 2020 1 commit
  2. 27 May, 2020 1 commit
  3. 22 May, 2020 1 commit
  4. 16 Mar, 2020 1 commit
  5. 28 Feb, 2020 1 commit
  6. 26 Feb, 2020 1 commit
    • Geoff Brown's avatar
      Bug 1614640 - Improve reftest suite_start data to provide consistent... · 5fc9f151
      Geoff Brown authored
      Bug 1614640 - Improve reftest suite_start data to provide consistent group/manifest names even when not run-by-manifest; r=ahal
      
      Reftest-family test suites can be "run-by-manifest" (reftest-plain, on desktop), or not
      (all reftest-family suites on Android, crashtest and jsreftest on desktop). When not
      run-by-manifest, code in reftest.jsm is used to format the suite_start data; when
      run-by-manifest, code in runreftest.py is used. Currently, the reftest.jsm implementation
      submits a simple array of test IDs -- no associated manifests -- possibly consistent with
      the concept that tests are not being run/grouped by manifest.
      
      This patch updates the reftest.jsm implementation to generate the same format of
      suite_start data as the python implementation: An array of dictionaries mapping
      manifest IDs to arrays of associated test IDs.
      
      Differential Revision: https://phabricator.services.mozilla.com/D63875
      
      --HG--
      extra : moz-landing-system : lando
      5fc9f151
  7. 09 Mar, 2020 1 commit
    • Andrew Halberstadt's avatar
      Bug 1325207 - [reftest] Stop treating 'skip/skip-if' as a failure type in the manifests r=kats · a9dd0b2e
      Andrew Halberstadt authored
      Currently the RefTest manifest parser has 5 possible statuses:
      
      0 => EXPECTED_PASS
      1 => EXPECTED_FAIL
      2 => EXPECTED_RANDOM
      3 => EXPECTED_DEATH   (aka skip)
      4 => EXPECTED_FUZZY
      
      In the manifests, the last status annotation that appears on the line will take
      precedence. For example:
      
          skip-if(true) fails-if(true) == test1.html ref.html
          fails-if(true) skip-if(true) == test2.html ref.html
      
      The first test will have an expected status equal to EXPECTED_FAIL, whereas the
      second one will be EXPECTED_DEATH. The same holds true for any combination of
      'fail/random/skip/fuzzy' annotations. This means developers need to be very
      careful about the order they specify these annotations as getting the order
      wrong can easily lead to unexpected behaviour.
      
      With the introduction of defaults in bug 1616368, the risk of unexpected behaviour
      is far greater. Since defaults are simply prepended to the test line, a manifest
      that looks like:
      
          defaults skip-if(true)
          == test1.html ref.html
          fails-if(true) == test2.html ref.html
      
      will actually skip the first test, but run the second (since the fails-if
      overwrites EXPECTED_DEATH with EXPECTED_FAIL).
      
      The root of the problem appears to be that 'skip' and 'fuzzy' are not actually
      test statuses. They are modifiers that affect how we run the test, but don't
      actually affect whether the test is expected to pass or fail.
      
      Therefore, this patch solves the problem by making 'skip/skip-if' its own thing
      that does not get overwritten by other failure types. In otherwords, a 'skip-if'
      can appear before or after a 'fails-if' and it will have the same meaning.
      
      Differential Revision: https://phabricator.services.mozilla.com/D64457
      
      --HG--
      extra : moz-landing-system : lando
      a9dd0b2e
  8. 26 Feb, 2020 1 commit
  9. 09 Jan, 2020 1 commit
  10. 09 Dec, 2019 1 commit
  11. 04 Dec, 2019 1 commit
  12. 03 Dec, 2019 1 commit
  13. 04 Dec, 2019 1 commit
  14. 27 Nov, 2019 1 commit
  15. 22 Nov, 2019 1 commit
  16. 18 Nov, 2019 3 commits
    • Timothy Nikkel's avatar
      Bug 1593170. Make the reftest-content.js functions FlushRendering and... · 75875025
      Timothy Nikkel authored
      Bug 1593170. Make the reftest-content.js functions FlushRendering and SynchronizeForSnapshot work on Fission child oop iframes. r=mattwoodrow,kmag
      
      This changes them to return a promise that resolves when the work is done, but we still need to change the callers to handle this new return type and do the right thing when these functions do their work async-ly.
      
      To do this we add a JSWindowActor called ReftestFission. reftest-content.js communicates with this actor via reftest.jsm.
      
      Differential Revision: https://phabricator.services.mozilla.com/D51343
      
      --HG--
      extra : moz-landing-system : lando
      75875025
    • Arthur Iakab's avatar
      Backed out 8 changesets (bug 1593170) for causing mass reftest failures. CLOSED TREE · 3e63c294
      Arthur Iakab authored
      Backed out changeset 2c98625f235b (bug 1593170)
      Backed out changeset 39d63ae4d287 (bug 1593170)
      Backed out changeset 2c8e57a90cb8 (bug 1593170)
      Backed out changeset d511975b764e (bug 1593170)
      Backed out changeset a10072d821e5 (bug 1593170)
      Backed out changeset 80bd8cadf835 (bug 1593170)
      Backed out changeset a004de649342 (bug 1593170)
      Backed out changeset 78d380a2241a (bug 1593170)
      3e63c294
    • Timothy Nikkel's avatar
      Bug 1593170. Make the reftest-content.js functions FlushRendering and... · c8fc6a9a
      Timothy Nikkel authored
      Bug 1593170. Make the reftest-content.js functions FlushRendering and SynchronizeForSnapshot work on Fission child oop iframes. r=mattwoodrow,kmag
      
      This changes them to return a promise that resolves when the work is done, but we still need to change the callers to handle this new return type and do the right thing when these functions do their work async-ly.
      
      To do this we add a JSWindowActor called ReftestFission. reftest-content.js communicates with this actor via reftest.jsm.
      
      Differential Revision: https://phabricator.services.mozilla.com/D51343
      
      --HG--
      extra : moz-landing-system : lando
      c8fc6a9a
  17. 04 Nov, 2019 1 commit
  18. 10 Oct, 2019 1 commit
    • Geoff Brown's avatar
      Bug 1521640 - Track assertion counts in reftest harness; r=jgraham · 089eb143
      Geoff Brown authored
      These assertion counts were removed (accidentally?) by bug 1321127, effectively
      not tracking assertion count mismatches in the reftest harness and instead
      relying on mozharness to fail tasks based on the logged error messages.
      Restoring the counts ensures:
       - the reftest summary includes accurate assertion counts like
         REFTEST INFO | Unexpected: 12 (..., 11 unexpected asserts, ...)
         REFTEST INFO | Known problems: 64 (..., 31 known asserts, ...)
       - assertion mismatches cause the harness to exit with an error code so
         that the job fails even if the log parsing is broken (bug 1587139)
         or the tests are being run locally with mach.
      
      Differential Revision: https://phabricator.services.mozilla.com/D48594
      
      --HG--
      extra : moz-landing-system : lando
      089eb143
  19. 20 Sep, 2019 2 commits
  20. 04 Jul, 2019 2 commits
    • Kartikaya Gupta's avatar
      Bug 1525314 - Auto-fuzz WR on Android with maxDifference<=2. r=jmaher · a8981111
      Kartikaya Gupta authored
      Due to the sheer number of tests that exhibit a random fuzz with maxDifference=1
      and maxDifference=2 with WR on Android, it's easier to just tweak the harness
      to autofuzz these away. This adds machinery to do so, and also adds a new
      annotation that can be used to disable the autofuzzing on specific tests.
      
      Depends on D36794
      
      Differential Revision: https://phabricator.services.mozilla.com/D36796
      
      --HG--
      extra : moz-landing-system : lando
      a8981111
    • Coroiu Cristina's avatar
      Backed out 6 changesets (bug 1525314) for reftest failures at... · 3983fddf
      Coroiu Cristina authored
      Backed out 6 changesets (bug 1525314) for reftest failures at reftests/svg/filters/css-filters/saturate-zero.html om a CLOSED TREE
      
      Backed out changeset 0ed2509b7191 (bug 1525314)
      Backed out changeset af72d1c4c107 (bug 1525314)
      Backed out changeset ab21a3ff4ae4 (bug 1525314)
      Backed out changeset 02399933ac4b (bug 1525314)
      Backed out changeset 95790a07a93c (bug 1525314)
      Backed out changeset 28f52fd3934e (bug 1525314)
      3983fddf
  21. 03 Jul, 2019 1 commit
  22. 24 Jun, 2019 1 commit
  23. 05 Jun, 2019 1 commit
  24. 19 Nov, 2018 1 commit
  25. 06 Nov, 2018 1 commit
    • Geoff Brown's avatar
      Bug 1453895 - Avoid reftest startup hang waiting for focus; r=jmaher · 7fdf08e2
      Geoff Brown authored
      Intermittently, at least on Windows, reftest.jsm finds that the reftest.xul
      window is not focused on startup; focus seems to be on the dummy about:blank
      window. This patch tries to detect that condition and correct it by bringing
      reftest.xul (g.containingWindow) into focus before specifically focusing
      the browser element (g.browser).
      7fdf08e2
  26. 01 Oct, 2018 1 commit
  27. 24 Sep, 2018 1 commit
  28. 06 Sep, 2018 1 commit
  29. 24 Jul, 2018 1 commit
  30. 17 Jul, 2018 1 commit
  31. 12 Jul, 2018 1 commit
  32. 14 Jun, 2018 1 commit
  33. 18 Jun, 2018 1 commit
  34. 07 Jun, 2018 1 commit
  35. 20 May, 2018 1 commit
  36. 18 May, 2018 1 commit