Commit e7c5c25a authored by Andrew Creskey's avatar Andrew Creskey
Browse files

Bug 1835408 - Add an http/3 upload performance test r=perftest-reviewers,afinder,sparky 

We want to track h3 upload performance so that we can improve it.

Differential Revision: https://phabricator.services.mozilla.com/D179276
parent 53f87d35

taskcluster/ci/test/browsertime-desktop.yml

+4 −2
Original line number Diff line number Diff line
@@ -701,8 +701,10 @@ browsertime-regression-tests: 
browsertime-upload:

    raptor:

        apps: [firefox, chrome, chromium, safari]

        subtests: [upload]

    description: Raptor (browsertime) file upload performance test

        subtests:

            - [upload, upload-h2]

            - upload-h3

    description: Raptor (browsertime) http2, http3 file upload performance test

    max-run-time: 2700

    run-on-projects:

        by-app:

testing/perfdocs/generated/raptor.rst

+233 −0
Original line number Diff line number Diff line
@@ -8891,6 +8891,239 @@ Browsertime tests that use a custom pageload test script. These use the pageload 
























.. dropdown:: upload-h3









   :class-container: anchor-id-upload-h3-c

















   **Owner**: Network Team

















   * **alert threshold**: 2.0









   * **apps**: firefox, chrome, chromium, safari









   * **browser cycles**: 1









   * **browsertime args**: --browsertime.upload_iterations=10 --firefox.preference=network.http.http3.enable:true









   * **custom data**: true









   * **expected**: pass









   * **gecko profile entries**: 14000000









   * **gecko profile interval**: 1









   * **lower is better**: false









   * **output timeout**: 2000000









   * **page cycles**: 1









   * **page timeout**: 1800000









   * **subtest lower is better**: false









   * **subtest name filters**: tests/,iterations/









   * **subtest unit**: mbps









   * **test script**: upload.js









   * **test url**: `<None>`__









   * **type**: pageload









   * **unit**: mbps









   * **use live sites**: true









   * **Test Task**:

















   .. list-table:: **test-linux1804-64-nightlyasrelease-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-firefox-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-linux1804-64-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-firefox-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-linux1804-64-shippable-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-chrome-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-chromium-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-firefox-upload-h3**









        - ✅









        - ✅









        - ❌









        - ❌

























   .. list-table:: **test-macosx1015-64-nightlyasrelease-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-firefox-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-macosx1015-64-shippable-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-chrome-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-chromium-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-firefox-upload-h3**









        - ✅









        - ✅









        - ❌









        - ❌









      * - **browsertime-upload-safari-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-windows10-32-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-firefox-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-windows10-32-shippable-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-chrome-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-chromium-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-firefox-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-windows10-64-nightlyasrelease-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-firefox-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-windows10-64-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-firefox-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌

























   .. list-table:: **test-windows10-64-shippable-qr/opt**









      :widths: 30 15 15 15 15









      :header-rows: 1

















      * - **Test Name**









        - mozilla-central









        - autoland









        - mozilla-release









        - mozilla-beta









      * - **browsertime-upload-chrome-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-chromium-upload-h3**









        - ❌









        - ❌









        - ❌









        - ❌









      * - **browsertime-upload-firefox-upload-h3**









        - ✅









        - ✅









        - ❌









        - ❌

































.. dropdown:: welcome









   :class-container: anchor-id-welcome-c








































testing/raptor/browsertime/upload.js



+1
−1






























Original line number
Diff line number
Diff line







@@ -51,7 +51,7 @@ module.exports = async function (context, commands) {

















    if (context.options.browsertime.moz_fetch_dir == "None") {









      context.log.error(









        "This test depends on the fetch task. Download the file, 'https://github.com/mozilla/perf-automation/raw/master/test_files/upload-test-50MB.dat' and set the os environment variable MOZ_FETCHES_DIR to that directory."









        "This test depends on the fetch task. Download the file, 'https://github.com/mozilla/perf-automation/raw/master/test_files/upload-test-32MB.dat' and set the os environment variable MOZ_FETCHES_DIR to that directory."









      );









    }










































testing/raptor/raptor/perfdocs/config.yml



+2
−1






























Original line number
Diff line number
Diff line







@@ -163,7 +163,8 @@ suites:







            welcome: "Measures pageload metrics for the first-install about:welcome page"









            grandprix: "Runs the Grandprix benchmark"









            constant-regression: "Generates a constant value that can be changed to induce a regression."









            upload: "Measures file upload speed"









            upload: "Measures http/2 file upload throughput"









            upload-h3: "Measures http/3 file upload throughput"









            sample-python-support: "A sample test that uses a python support file to modify the test command."









    interactive:









        description: "Browsertime tests that interact with the webpage. Includes responsiveness tests as they make use of this support for navigation. These form of tests allow the specification of browsertime commands through the test manifest."
































testing/raptor/raptor/tests/custom/browsertime-upload.ini



+12
−1






























Original line number
Diff line number
Diff line







@@ -18,8 +18,9 @@ type = pageload







unit = mbps









use_live_sites = true



















# upload performance custom test









# upload performance custom tests



















# http/2









[upload]









browsertime_args = --browsertime.upload_iterations=10 --firefox.preference=network.http.http3.enable:false --chrome.args disable-quic









custom_data = true








@@ -28,3 +29,13 @@ lower_is_better = false







subtest_name_filters = tests/,iterations/









test_script = upload.js









test_url = None



















# http/3









[upload-h3]









browsertime_args = --browsertime.upload_iterations=10 --firefox.preference=network.http.http3.enable:true









custom_data = true









unit = mbps









lower_is_better = false









subtest_name_filters = tests/,iterations/









test_script = upload.js









test_url = None