# glean pings

If data collection is enabled, glean provides a set of pings that are assembled out of the box
without any developer intervention.
Every glean ping is in JSON format and contains a [common section](#the-ping_info-section)
with shared information data included under the `ping_info` key.
The following is a list of the pings along with the conditions that triggers them.

- [`baseline` ping](baseline.md)
- [`events` ping](events.md)
- [`metrics` ping](metrics.md)

## The `ping_info` section
The following fields are always included in the `ping_info` section, for every ping.

| Field name | Type | Description |
|---|---|---|
| `ping_type` | String | The name of the ping type (e.g. "baseline", "metrics") |
| `app_build` | String | The build identifier generated by the CI system (e.g. "1234/A") |
| `app_display_version` | String | The user-visible version string (e.g. "1.0.3") |
| `telemetry_sdk_build` | String | The version of the glean library |
| `client_id` | UUID |  A UUID identifying a profile and allowing user-oriented correlation of data |
| `seq` | Counter | A running counter of the number of times pings of this type have been sent |
| `experiments` | Object | A dictionary of [active experiments](#the-experiments-object) |
| `start_time` | Datetime | The time of the start of collection of the data in the ping, in local time and with minute precision. | See [note](#a-note-about-time-formats) |
| `end_time` | Datetime | The time of the end of collection of the data in the ping, in local time and with minute precision. This is also the time this ping was generated and is likely well before ping transmission time | See [note](#a-note-about-time-formats) |
| `first_run_date` | Datetime | The date of the first run of the application, in local time and with day precision. | See [note](#a-note-about-time-formats)  |

All the metrics surviving application restarts (e.g. `client_id`, `seq`, ...) are removed once the
application using glean is uninstalled.

### A note about time formats
Time formats are used and expected in ISO 8601 format.  Due to the minimum Android SDK version 21
not having direct support of outputting or parsing these formats, there was difficulty in finding
a way to output these formats with the `:` character properly encoded in the timezone offset.  So
we get the following:

```
2018-12-19T12:36-0600
```

We require the following to comply with certain back-end services:

```
2018-12-19T12:36-06:00
```

Which is why the `:` is manually inserted in order to comply with the format and requirements.

### The `experiments` object
This object contains experiments keyed by the experiment `id`. Each listed experiment contains the
`branch` the client is enrolled in and may contain a string to string map with additional data in the
`extra` key. Both the `id` and `branch` are truncated to 30 characters.

```json
{
  "<id>": {
    "branch": "branch-id",
    "extra": {
      "some-key": "a-value"
    }
  }
}
```

## Ping submission
The pings glean generates are submitted to the Mozilla servers at specific paths, in order to provide
additional metadata without the need of unpacking the ping payload. A typical submission URL looks like
`"<server-address>/submit/<application-id>/<doc-type>/<glean-schema-version>/<ping-uuid>"`, with:

- `<server-address>`: the address of the server that receives the pings;
- `<application-id>`: a unique application id, automatically detected by glean; this is the value returned by [`Context.getPackageName()`](http://developer.android.com/reference/android/content/Context.html#getPackageName());
- `<doc-type>`: the name of the ping; this can be one of the pings available out of the box with glean, or a custom ping;
- `<glean-schema-version>`: the version of the glean ping schema;
- `<ping-uuid>`: a unique identifier for this ping.

### Submitted headers
A pre-defined set of headers is additionally sent along with the submitted ping:

| Header | Value | Description |
|--------|-------|-------------|
| `Content-Type` | `application/json; charset=utf-8` | Describes the data sent to the server |
| `User-Agent` | Defaults to e.g. `Glean/0.40.0 (Android)`, where `0.40.0` is the glean version number and `Android` is the platform name. It can be overriden by user through [configuration](https://github.com/mozilla-mobile/android-components/blob/8441728f156847a58dcb25d29254512a1801b266/components/service/glean/src/main/java/mozilla/components/service/glean/config/Configuration.kt#L23) | Describes the application sending the ping using glean |
| `Date` | e.g. `Mon, 23 Jan 2019 10:10:10 GMT+00:00` | Submission date/time in GMT/UTC+0 offset |
| `X-Client-Type` | `Glean` | Custom header to support handling of glean pings in the legacy pipeline |
| `X-Client-Version` | e.g. `0.40.0` | The glean version, sent as a custom header to support handling of glean pings in the legacy pipeline |

## A note about string length
In order to ensure that pings don't become accidentally large, most string
values are automatically truncated. The current implementation truncates based
on a maximum number of Unicode characters (defined in the API docs), since is
the most expedient implementation-wise. Other or future implementations may
perform this truncation based on the number of UTF8 bytes instead, which is
ultimately what matters for ping size.

## Defining background state
These docs refer to application 'background' state in several places. This specifically means when 
the activity is no longer visible to the user, it has entered the Stopped state, and the system 
invokes the [onStop()](https://developer.android.com/reference/android/app/Activity.html#onStop()) callback. 
This may occur, for example, when a newly launched activity covers the entire screen. The system may 
also call onStop() when the activity has finished running, and is about to be terminated.

## Error reporting

Glean records the number of errors that occur when metrics are passed invalid
data or are otherwise used incorrectly. This information is reported back in
special labeled counter metrics in the `glean.error` category. Error metrics are
included in the same pings as the metric that caused the error. Additionally,
error metrics are always sent in the [`metrics` ping](metrics.md) ping.

The following categories of errors are recorded:

- `invalid_value`: The metric value was invalid or out-of-range.
- `invalid_label`: The label on a labeled metric was invalid.

For example, if you had a string metric and passed it a string that was too long:

```
MyMetrics.stringMetric.set("this_string_is_longer_than_the_limit_for_string_metrics")
```

The following error metric counter would be incremented:

```
Glean.error.invalidValue.add(1)
```

Resulting in the following keys in the ping:

```
{
  "metrics": {
    "labeled_counter": {
      "glean.error.invalid_value": {
        "my_metrics.string_metric": 1
      }
    }
  }
}
```

If you have a debug build of glean, details about the errors being recorded are
included in the logs. This detailed information is not included in glean pings.

