Changes
Page history
fix typos, stop putting blackbox in pre everywhere
authored
Oct 03, 2024
by
anarcat
I was doing this to bypass my spellchecker, but that's dumb, added it to a file-local list instead.
Show whitespace changes
Inline
Side-by-side
service/prometheus.md
View page @
4fa62cd4
...
@@ -331,7 +331,7 @@ servers.
...
@@ -331,7 +331,7 @@ servers.
[
prometheus-alerts
]:
https://gitlab.torproject.org/tpo/tpa/prometheus-alerts
[
prometheus-alerts
]:
https://gitlab.torproject.org/tpo/tpa/prometheus-alerts
Note: we currently have a handful of
`blackbox
-
exporter`
-related targets for TPA
Note: we currently have a handful of
`blackbox
_
exporter`
-related targets for TPA
services, namely for the HTTP checks. We intend to move those into puppet
services, namely for the HTTP checks. We intend to move those into puppet
profiles whenever possible.
profiles whenever possible.
...
@@ -384,7 +384,7 @@ Those rules are declared on the server, in `prometheus::prometheus::server::inte
...
@@ -384,7 +384,7 @@ Those rules are declared on the server, in `prometheus::prometheus::server::inte
[
`gitlab#20`
]:
https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/20
[
`gitlab#20`
]:
https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/20
### Adding a
`
blackbox
`
target
### Adding a blackbox target
Most exporters are pretty straightforward: a service binds to a port and exposes
Most exporters are pretty straightforward: a service binds to a port and exposes
metrics through HTTP requests on that port, generally on the
`/metrics`
URL.
metrics through HTTP requests on that port, generally on the
`/metrics`
URL.
...
@@ -399,16 +399,16 @@ will try to reach. The check will be initiated from the host running the
...
@@ -399,16 +399,16 @@ will try to reach. The check will be initiated from the host running the
blackbox exporter to the target at the moment the Prometheus server is scraping
blackbox exporter to the target at the moment the Prometheus server is scraping
the exporter.
the exporter.
The
`
blackbox
_
exporter
`
is rather peculiar and counter-intuitive, see
The blackbox
exporter is rather peculiar and counter-intuitive, see
the
[
how to debug the
`
blackbox
_
exporter
`
](
#debugging-blackbox
_
exporter
)
for
the
[
how to debug the blackbox
exporter
](
#debugging-blackbox
-
exporter
)
for
more information.
more information.
#### Scrape jobs
#### Scrape jobs
In Prometheus's point of view, two information
s
are needed:
In Prometheus's point of view, two information are needed:
*
t
he address and port of the host where Prometheus can reach the blackbox exporter
*
T
he address and port of the host where Prometheus can reach the blackbox exporter
*
t
he target (and possibly the port tested) that the exporter will try to reach
*
T
he target (and possibly the port tested) that the exporter will try to reach
Prometheus transfers the information above to the exporter via two labels:
Prometheus transfers the information above to the exporter via two labels:
...
@@ -424,7 +424,7 @@ Prometheus transfers the information above to the exporter via two labels:
...
@@ -424,7 +424,7 @@ Prometheus transfers the information above to the exporter via two labels:
The reshuffling of labels mentioned above is achieved with the
`relabel_configs`
The reshuffling of labels mentioned above is achieved with the
`relabel_configs`
option for the scrape job.
option for the scrape job.
For TPA-managed services, we define this scrape jobs in
h
iera in
For TPA-managed services, we define this scrape jobs in
H
iera in
`common/prometheus.yml`
under keys named
`collect_scrape_jobs`
. Jobs in those
`common/prometheus.yml`
under keys named
`collect_scrape_jobs`
. Jobs in those
keys expect targets to be exported by other parts of the puppet code.
keys expect targets to be exported by other parts of the puppet code.
...
@@ -497,7 +497,7 @@ For non-TPA services, the targets need to be defined in the `prometheus-alerts`
...
@@ -497,7 +497,7 @@ For non-TPA services, the targets need to be defined in the `prometheus-alerts`
repository.
repository.
The targets defined this way for blackbox exporter look exactly like normal
The targets defined this way for blackbox exporter look exactly like normal
p
rometheus targets, except that they define what the blackbox exporter will try
P
rometheus targets, except that they define what the blackbox exporter will try
to reach. The targets can be
`hostname:port`
pairs or URLs, depending on the
to reach. The targets can be
`hostname:port`
pairs or URLs, depending on the
nature of the type of check being defined.
nature of the type of check being defined.
...
@@ -636,7 +636,7 @@ configuration, or alerting rule:
...
@@ -636,7 +636,7 @@ configuration, or alerting rule:
-
`job`
: name of the job (e.g.
`JobDown`
)
-
`job`
: name of the job (e.g.
`JobDown`
)
-
`instance`
: host name and port of affected device, including URL for
-
`instance`
: host name and port of affected device, including URL for
some
`
blackbox
`
probes (e.g.
`test-01.torproject.org:9100`
,
some blackbox probes (e.g.
`test-01.torproject.org:9100`
,
`https://www.torproject.org`
)
`https://www.torproject.org`
)
-
`alias`
: similar to instance, without the port number
-
`alias`
: similar to instance, without the port number
(e.g.
`test-01.torproject.org`
,
`https://www.torproject.org`
)
(e.g.
`test-01.torproject.org`
,
`https://www.torproject.org`
)
...
@@ -2050,7 +2050,7 @@ would otherwise be around long enough for Prometheus to scrape their
...
@@ -2050,7 +2050,7 @@ would otherwise be around long enough for Prometheus to scrape their
metrics. We use it as a workaround to bridge Metrics data with
metrics. We use it as a workaround to bridge Metrics data with
Prometheus/Grafana.
Prometheus/Grafana.
## Debugging
`
blackbox
_
exporter
`
## Debugging
the
blackbox
exporter
The
[
upstream documentation
][]
has some details that can help. We also
The
[
upstream documentation
][]
has some details that can help. We also
have examples
[
above
][]
for how to configure it in our setup.
have examples
[
above
][]
for how to configure it in our setup.
...
@@ -2064,7 +2064,7 @@ want to include debugging output. For example, to run an ICMP test on host
...
@@ -2064,7 +2064,7 @@ want to include debugging output. For example, to run an ICMP test on host
curl http://localhost:9115/probe?target=pauli.torproject.org&module=icmp&debug=true
curl http://localhost:9115/probe?target=pauli.torproject.org&module=icmp&debug=true
Note that the above trick can be used for _any_ target, not just for ones
Note that the above trick can be used for _any_ target, not just for ones
currently configured in the
`
blackbox
_
exporter
`
. So you can also use this to test
currently configured in the blackbox
exporter. So you can also use this to test
things before creating the final configuration for the target.
things before creating the final configuration for the target.
[
upstream documentation
]:
https://github.com/prometheus/blackbox_exporter
[
upstream documentation
]:
https://github.com/prometheus/blackbox_exporter
...
...
...
...