Document how to use the fabric tasks for creating/updating silences authored by lelutin's avatar lelutin
......@@ -177,6 +177,45 @@ criteria) and then clicking on the bell icon at the top, just right of the
filter bar. This method can help you create a silence for more than just one
alert at a time.
### Adding and updating silences with fabric
You can use [Fabric][] to manage silences from the command line or via scripts.
This is mostly useful for automatically adding a silence from some other,
higher-level tasks. But you can use the fabric task either directly or in other
scripts if you'd like.
Here's an example for adding a new silence for all backup alerts for the host
idle-dal-02.torproject.org with author "wario" and a comment:
fab silence.create --comment="machine waiting for first backup" \
--matchers job=bacula --matchers alias=idle-dal-02.torproject.org \
--ends-at "in 5 days" --created-by "wario"
The comment and the author are optional, but it is strongly recommended to set
them, so that others can understand the goal of the silence and can refer to you
for questions.
The matchers option can be specified multiple times. All values of the matchers
option must match for the silence to find alerts (so the values have an "and"
type boolean relationship)
starts-at is not specified in the example above and that implies that the
silence starts from now. You can use starts-at for example for planning a
silence that will only take effect at the start of a planned maintenance window
in the future.
starts-at and ends-at both accept either ISO 8601 formatted dates or textual
dates accepted by the [dateparser](https://pypi.org/project/dateparser/) python
module.
Finally, if you want to update a silence, the command is slightly different but
the arguments are the same, except for one addition `silence-id` which specifies
the ID of the alert that needs to be modified:
fab silence.update --comment="machine waiting for first backup - tpa/tpa/team#12345678" \
--matchers job=bacula --matchers alias=idle-dal-02.torproject.org \
--ends-at "in 7 days" --created-by "wario"
## Adding metrics to applications
If you want your service to be monitored by Prometheus, you need to
......
......