... | ... | @@ -121,10 +121,51 @@ See also the [targets.d documentation in the git repository](https://gitlab.torp |
|
|
|
|
|
## Adding targets on the internal server
|
|
|
|
|
|
TODO: talk about `scrape_jobs` for in-puppet configurations.
|
|
|
Normally, services configured in Puppet SHOULD automatically be
|
|
|
scraped by Prometheus (see below). If, however, you need to manually
|
|
|
configure a service, you *may* define extra jobs in the
|
|
|
`$scrape_configs` array, in the
|
|
|
`profile::prometheus::server::internal` Puppet class.
|
|
|
|
|
|
For example, because the GitLab Prometheus setup is not managed by
|
|
|
Puppet ([tpo/tpa/gitlab#20](https://gitlab.torproject.org/tpo/tpa/gitlab/-/issues/20)), we cannot use this automatic setup,
|
|
|
so manual scrape targets are defined like this:
|
|
|
|
|
|
$scrape_configs =
|
|
|
[
|
|
|
{
|
|
|
'job_name' => 'gitaly',
|
|
|
'static_configs' => [
|
|
|
{
|
|
|
'targets' => [
|
|
|
'gitlab-02.torproject.org:9236',
|
|
|
],
|
|
|
'labels' => {
|
|
|
'alias' => 'Gitaly-Exporter',
|
|
|
},
|
|
|
},
|
|
|
],
|
|
|
},
|
|
|
[...]
|
|
|
]
|
|
|
|
|
|
But ideally those would be configured with automatic targets, below.
|
|
|
|
|
|
## Automatic targets on the internal server
|
|
|
|
|
|
Metrics for the internal server are scraped automatically if the
|
|
|
exporter is configured by the [puppet-prometheus][] module. This is
|
|
|
done almost automatically, apart from the need to open a firewall port
|
|
|
in our configuration.
|
|
|
|
|
|
To take the `apache_exporter`, as an example, in
|
|
|
`profile::prometheus::apache_exporter`, include the
|
|
|
`prometheus::apache_exporter` class from the upstream Puppet module,
|
|
|
then we open the port to the Prometheus server on the exporter, with:
|
|
|
|
|
|
Ferm::Rule <<| tag == 'profile::prometheus::server-apache-exporter' |>>
|
|
|
|
|
|
TODO: show how to hook a custom scrape job, and on where server to put
|
|
|
it.
|
|
|
Those rules are declared on the server, in `prometheus::prometheus::server::internal`.
|
|
|
|
|
|
## Web dashboard access
|
|
|
|
... | ... | @@ -723,13 +764,14 @@ but it was [salvaged](https://github.com/prometheus-community/bind_exporter/issu |
|
|
|
|
|
Another important layer is the large amount of Puppet code that is
|
|
|
used to deploy Prometheus and its components. This is all part of a
|
|
|
big Puppet module, [puppet-prometheus](https://github.com/voxpupuli/puppet-prometheus/), managed by the [voxpupuli
|
|
|
big Puppet module, [puppet-prometheus][], managed by the [voxpupuli
|
|
|
collective](https://github.com/voxpupuli). Our integration with the module is not yet complete:
|
|
|
we have a lot of glue code on top of it to correctly make it work with
|
|
|
Debian packages. A lot of work has been done to complete that work by
|
|
|
anarcat, but work still remains, see [upstream issue 32](https://github.com/voxpupuli/puppet-prometheus/issues/32) for
|
|
|
details.
|
|
|
|
|
|
[puppet-prometheus]: https://github.com/voxpupuli/puppet-prometheus/
|
|
|
## Monitoring and testing
|
|
|
|
|
|
Prometheus doesn't have specific tests, but there *is* a test suite in
|
... | ... | |