Unverified Commit e4c1cd94 authored by anarcat's avatar anarcat
Browse files

some nagios diagnostics

parent fd41a297
Loading
Loading
Loading
Loading
+45 −0
Original line number Diff line number Diff line
@@ -18,3 +18,48 @@ Nagios/Icinga service for Tor Project infrastructure
## How to add a host

- Add new hosts to `tor-nagios/config/nagios-master.cfg`.

## Forcing a rebuild of the configuration

If the Nagios configuration seems out of sync with the YAML config, a
rebuild of the configuration can be forced with this command on the
Nagios server:

    touch /home/nagiosadm/tor-nagios/config/nagios-master.cfg && sudo -u nagiosadm make -C /home/nagiosadm/tor-nagios/config

Alternatively, changing the `.cfg` file and pushing a new commit
should trigger this as well.

# Reference

## Design

### Config generation

The Nagios/Icinga configuration gets generated from the
`config/nagios-master.cfg` YAML configuration file stored in the
`admin/tor-nagios.git` repository. The generation works like this:

 1. the [git server](git) has a post-receive hook (in
    `/srv/git.torproject.org/git-helpers/post-receive-per-repo.d/admin%tor-nagios/trigger-nagios-build`)

 2. ... which launches a "trigger" on the Nagios server, like so:

        ssh -i ~/.ssh/gitweb -l nagiosadm hetzner-hel1-01 -- -trigger-

 3. that SSH key, deployed from Puppet (so in
    `/etc/ssh/puppetkeys/nagiosadm`), calls the
    `/home/nagiosadm/bin/from-git-rw` which then...

 4. creates or updates (`git clone` or `git pull`) the git repository
    in `~/tor-nagios/config`...
    
 5. then calls `make` in the directory, which calls `./build-nagios`
    to generate the files in `~/tor-nagios/config/generated/`

 7. then calls `make install` in the `config` directory, which deploys
    the config file (using `rsync`) in `/etc/inciga/from-git` and also
    pushes the NRPE config to the [Puppet server](puppet) in
    `nagiospush@pauli.torproject.org:/etc/puppet/modules/nagios/files/tor-nagios/generated/nrpe_tor.cfg`

 8. then finally reloads incinga