Commit afbc8983 authored by Chris H-C's avatar Chris H-C
Browse files

Bug 1585410 - Implement and document 'deletion-request' ping r=janerik,Dexter 

The 'deletion-request' ping, which supercedes the 'optout' ping, notifies the
pipeline when a profile opts out of FHR upload. (IOW, when a user on a specific
profile unchecks the box in about:preferences#privacy about sharing
technical and interaction data with Mozilla).

This ping tries its best to reach the pipeline to let them know that we need
to delete data associated with the provided clientId. This means it will remain
pending on the client even after opt out and it will try to resend if upload is
ever re-enabled.

Differential Revision: https://phabricator.services.mozilla.com/D51710

--HG--
rename : toolkit/components/telemetry/docs/data/optout-ping.rst => toolkit/components/telemetry/docs/obsolete/optout-ping.rst
rename : toolkit/components/telemetry/tests/marionette/tests/client/test_optout_ping.py => toolkit/components/telemetry/tests/marionette/tests/client/test_deletion_request_ping.py
extra : moz-landing-system : lando
parent 533e2b3a

toolkit/components/telemetry/app/TelemetryController.jsm

+27 −9
Original line number Diff line number Diff line
@@ -45,7 +45,7 @@ const NEWPROFILE_PING_DEFAULT_DELAY = 30 * 60 * 1000; 


// Ping types.

const PING_TYPE_MAIN = "main";

const PING_TYPE_OPTOUT = "optout";

const PING_TYPE_DELETION_REQUEST = "deletion-request";



// Session ping reasons.

const REASON_GATHER_PAYLOAD = "gather-payload";
@@ -200,6 +200,8 @@ var TelemetryController = Object.freeze({ 
   *                  environment data.

   * @param {Object}  [aOptions.overrideEnvironment=null] set to override the environment data.

   * @param {Boolean} [aOptions.usePingSender=false] if true, send the ping using the PingSender.

   * @param {String} [aOptions.overrideClientId=undefined] if set, override the

   *                 client id to the provided value. Implies aOptions.addClientId=true.

   * @returns {Promise} Test-only - a promise that resolves with the ping id once the ping is stored or sent.

   */

  submitExternalPing(aType, aPayload, aOptions = {}) {
@@ -233,6 +235,8 @@ var TelemetryController = Object.freeze({ 
   * @param {Boolean} [aOptions.overwrite=false] true overwrites a ping with the same name,

   *                  if found.

   * @param {Object}  [aOptions.overrideEnvironment=null] set to override the environment data.

   * @param {String} [aOptions.overrideClientId=undefined] if set, override the

   *                 client id to the provided value. Implies aOptions.addClientId=true.

   *

   * @returns {Promise} A promise that resolves with the ping id when the ping is saved to

   *                    disk.
@@ -379,6 +383,8 @@ var Impl = { 
   * @param {Boolean} aOptions.addEnvironment true if the ping should contain the

   *                  environment data.

   * @param {Object}  [aOptions.overrideEnvironment=null] set to override the environment data.

   * @param {String} [aOptions.overrideClientId=undefined] if set, override the

   *                 client id to the provided value. Implies aOptions.addClientId=true.

   *

   * @returns {Object} An object that contains the assembled ping data.

   */
@@ -402,8 +408,8 @@ var Impl = { 
      payload,

    };



    if (aOptions.addClientId) {

      pingData.clientId = this._clientID;

    if (aOptions.addClientId || aOptions.overrideClientId) {

      pingData.clientId = aOptions.overrideClientId || this._clientID;

    }



    if (aOptions.addEnvironment) {
@@ -447,13 +453,16 @@ var Impl = { 
   *                  environment data.

   * @param {Object}  [aOptions.overrideEnvironment=null] set to override the environment data.

   * @param {Boolean} [aOptions.usePingSender=false] if true, send the ping using the PingSender.

   * @param {String} [aOptions.overrideClientId=undefined] if set, override the

   *                 client id to the provided value. Implies aOptions.addClientId=true.

   * @returns {Promise} Test-only - a promise that is resolved with the ping id once the ping is stored or sent.

   */

  async _submitPingLogic(aType, aPayload, aOptions) {

    // Make sure to have a clientId if we need one. This cover the case of submitting

    // a ping early during startup, before Telemetry is initialized, if no client id was

    // cached.

    if (!this._clientID && aOptions.addClientId) {

    if (!this._clientID && aOptions.addClientId && !aOptions.overrideClientId) {

      this._log.trace("_submitPingLogic - Waiting on client id");

      Telemetry.getHistogramById(

        "TELEMETRY_PING_SUBMISSION_WAITING_CLIENTID"

      ).add();
@@ -497,6 +506,8 @@ var Impl = { 
   *                  environment data.

   * @param {Object}  [aOptions.overrideEnvironment=null] set to override the environment data.

   * @param {Boolean} [aOptions.usePingSender=false] if true, send the ping using the PingSender.

   * @param {String} [aOptions.overrideClientId=undefined] if set, override the

   *                 client id to the provided value. Implies aOptions.addClientId=true.

   * @returns {Promise} Test-only - a promise that is resolved with the ping id once the ping is stored or sent.

   */

  submitExternalPing: function send(aType, aPayload, aOptions) {
@@ -559,6 +570,8 @@ var Impl = { 
   *                  environment data.

   * @param {Boolean} aOptions.overwrite true overwrites a ping with the same name, if found.

   * @param {Object}  [aOptions.overrideEnvironment=null] set to override the environment data.

   * @param {String} [aOptions.overrideClientId=undefined] if set, override the

   *                 client id to the provided value. Implies aOptions.addClientId=true.

   *

   * @returns {Promise} A promise that resolves with the ping id when the ping is saved to

   *                    disk.
@@ -1002,7 +1015,7 @@ var Impl = { 


  /**

   * Called whenever the FHR Upload preference changes (e.g. when user disables FHR from

   * the preferences panel), this triggers sending the optout ping.

   * the preferences panel), this triggers sending the "deletion-request" ping.

   */

  _onUploadPrefChange() {

    const uploadEnabled = Services.prefs.getBoolPref(
@@ -1050,13 +1063,18 @@ var Impl = { 
        TelemetrySession.resetSubsessionCounter();



        // 5. Set ClientID to a known value

        let oldClientId = this._clientID;

        this._clientID = await ClientID.setClientID(

          TelemetryUtils.knownClientID

        );



        // 6. Send the optout ping.

        this._log.trace("_onUploadPrefChange - Sending optout ping.");

        this.submitExternalPing(PING_TYPE_OPTOUT, {}, { addClientId: false });

        // 6. Send the deletion-request ping.

        this._log.trace("_onUploadPrefChange - Sending deletion-request ping.");

        this.submitExternalPing(

          PING_TYPE_DELETION_REQUEST,

          {},

          { overrideClientId: oldClientId }

        );

      }

    })();
@@ -1070,7 +1088,7 @@ var Impl = { 


  _attachObservers() {

    if (IS_UNIFIED_TELEMETRY) {

      // Watch the FHR upload setting to trigger optout pings.

      // Watch the FHR upload setting to trigger "deletion-request" pings.

      Services.prefs.addObserver(

        TelemetryUtils.Preferences.FhrUploadEnabled,

        this,

toolkit/components/telemetry/app/TelemetrySend.jsm

+19 −30
Original line number Diff line number Diff line
@@ -70,7 +70,7 @@ const IS_UNIFIED_TELEMETRY = Services.prefs.getBoolPref( 


const MS_IN_A_MINUTE = 60 * 1000;



const PING_TYPE_OPTOUT = "optout";

const PING_TYPE_DELETION_REQUEST = "deletion-request";



// We try to spread "midnight" pings out over this interval.

const MIDNIGHT_FUZZING_INTERVAL_MS = 60 * MS_IN_A_MINUTE;
@@ -133,12 +133,12 @@ function isV4PingFormat(aPing) { 
}



/**

 * Check if the provided ping is an optout ping.

 * Check if the provided ping is a deletion-request ping.

 * @param {Object} aPing The ping to check.

 * @return {Boolean} True if the ping is an optout ping, false otherwise.

 * @return {Boolean} True if the ping is a deletion-request ping, false otherwise.

 */

function isOptoutPing(aPing) {

  return isV4PingFormat(aPing) && aPing.type == PING_TYPE_OPTOUT;

function isDeletionRequestPing(aPing) {

  return isV4PingFormat(aPing) && aPing.type == PING_TYPE_DELETION_REQUEST;

}



/**
@@ -239,7 +239,7 @@ var TelemetrySend = { 
  /**

   * Check if sending is disabled. If Telemetry is not allowed to upload,

   * pings are not sent to the server.

   * If trying to send an optout ping, don't block it.

   * If trying to send a deletion-request ping, don't block it.

   *

   * @param {Object} [ping=null] A ping to be checked.

   * @return {Boolean} True if pings can be send to the servers, false otherwise.
@@ -486,7 +486,7 @@ var SendScheduler = { 
      }



      // Get a list of pending pings, sorted by last modified, descending.

      // Filter out all the pings we can't send now. This addresses scenarios like "optout" pings

      // Filter out all the pings we can't send now. This addresses scenarios like "deletion-request" pings

      // which can be sent even when upload is disabled.

      let pending = TelemetryStorage.getPendingPingList();

      let current = TelemetrySendImpl.getUnpersistedPings();
@@ -499,9 +499,9 @@ var SendScheduler = { 
      // Note that the two lists contain different kind of data. |pending| only holds ping

      // info, while |current| holds actual ping data.

      if (!TelemetrySendImpl.sendingEnabled()) {

        // If sending is disabled, only handle an unpersisted optout ping

        // If sending is disabled, only handle deletion-request pings

        pending = [];

        current = current.filter(p => isOptoutPing(p));

        current = current.filter(p => isDeletionRequestPing(p));

      }

      this._log.trace(

        "_doSendTask - can send - pending: " +
@@ -1083,19 +1083,11 @@ var TelemetrySendImpl = { 
        try {

          await this._doPing(ping, ping.id, false);

        } catch (ex) {

          if (isOptoutPing(ping)) {

            // Optout pings should only be tried once and then discarded.

            this._log.info(

              "sendPings - optout ping " + ping.id + " not sent, discarding",

              ex

            );

          } else {

          this._log.info(

            "sendPings - ping " + ping.id + " not sent, saving to disk",

            ex

          );

          await savePing(ping);

          }

        } finally {

          this._currentPings.delete(ping.id);

        }
@@ -1458,7 +1450,7 @@ var TelemetrySendImpl = { 
  /**

   * Check if sending is disabled. If Telemetry is not allowed to upload,

   * pings are not sent to the server.

   * If trying to send an optout ping, don't block it.

   * If trying to send a "deletion-request" ping, don't block it.

   * If unified telemetry is off, don't send pings if Telemetry is disabled.

   *

   * @param {Object} [ping=null] A ping to be checked.
@@ -1477,8 +1469,8 @@ var TelemetrySendImpl = { 
    // With unified Telemetry, the FHR upload setting controls whether we can send pings.

    // The Telemetry pref enables sending extended data sets instead.

    if (IS_UNIFIED_TELEMETRY) {

      // Optout pings are sent once even if the upload is disabled.

      if (ping && isOptoutPing(ping)) {

      // "deletion-request" pings are sent once even if the upload is disabled.

      if (ping && isDeletionRequestPing(ping)) {

        return true;

      }

      return Services.prefs.getBoolPref(
@@ -1523,11 +1515,8 @@ var TelemetrySendImpl = { 
  async _persistCurrentPings() {

    for (let [id, ping] of this._currentPings) {

      try {

        // Never save an optout ping to disk

        if (!isOptoutPing(ping)) {

        await savePing(ping);

        this._log.trace("_persistCurrentPings - saved ping " + id);

        }

      } catch (ex) {

        this._log.error("_persistCurrentPings - failed to save ping " + id, ex);

      } finally {

toolkit/components/telemetry/docs/concepts/pings.rst

+1 −1
Original line number Diff line number Diff line
@@ -26,4 +26,4 @@ Important examples are: 
* :doc:`crash <../data/crash-ping>` - a ping that is captured and sent after a Firefox process crashes.

* :doc:`new-profile <../data/new-profile-ping>` - sent on the first run of a new profile.

* :doc:`update <../data/update-ping>` - sent right after an update is downloaded.

* :doc:`optout <../data/optout-ping>` - sent when FHR upload is disabled
 
* :doc:`deletion-request <../data/deletion-request-ping>` - sent when FHR upload is disabled

toolkit/components/telemetry/docs/data/deletion-request-ping.rst

0 → 100644
+35 −0
Original line number Diff line number Diff line 
"deletion-request" ping

=======================



This ping is submitted when a user opts out of sending technical and interaction data to Mozilla.

(In other words, when the

``datareporting.healthreport.uploadEnabled``

:doc:`preference <../internals/preferences>` is set to ``false``.)



This ping contains the client id.

This ping does not contain any environment data.



Structure:



.. code-block:: js



    {

      version: 4,

      type: "deletion-request",

      ... common ping data (including clientId)

      payload: { }

    }



Expected behaviours

-------------------

The following is a list of expected behaviours for the ``deletion-request`` ping:



- Telemetry will try to send the ping even if upload is disabled.

- Telemetry may persist this ping if it can't be immediately sent, and may try to resend it later.



Version History

---------------



- Firefox 70:



  - "deletion-request" ping replaces the "optout" ping (`bug 1585410 <https://bugzilla.mozilla.org/show_bug.cgi?id=1585410>`_).

toolkit/components/telemetry/docs/data/index.rst

+0 −2
Original line number Diff line number Diff line
@@ -10,8 +10,6 @@ Data documentation 
   common-ping

   environment

   main-ping

   optout-ping

   deletion-ping

   crash-ping

   backgroundhangmonitor-ping

   anonymous-ping