Skip to content
Snippets Groups Projects
Verified Commit 2836159b authored by Jérôme Charaoui's avatar Jérôme Charaoui :telescope:
Browse files

howto/puppet: new puppetdb endpoint (team#41341)

parent 4e8a2058
No related branches found
No related tags found
No related merge requests found
......@@ -736,12 +736,12 @@ YAML file. See this [immerda snippet](https://code.immerda.ch/immerda/puppet-mod
This will list all active hosts known to the Puppet master:
ssh -t pauli.torproject.org 'sudo -u postgres psql puppetdb -P pager=off -A -t -c "SELECT c.certname FROM certnames c WHERE c.deactivated IS NULL"'
ssh -t puppetdb-01.torproject.org 'sudo -u postgres psql puppetdb -P pager=off -A -t -c "SELECT c.certname FROM certnames c WHERE c.deactivated IS NULL"'
The following will list all hosts under Puppet and their `virtual`
value:
ssh -t pauli.torproject.org "sudo -u postgres psql puppetdb -P pager=off -F',' -A -t -c \"SELECT c.certname, value_string FROM factsets fs INNER JOIN facts f ON f.factset_id = fs.id INNER JOIN fact_values fv ON fv.id = f.fact_value_id INNER JOIN fact_paths fp ON fp.id = f.fact_path_id INNER JOIN certnames c ON c.certname = fs.certname WHERE fp.name = 'virtual' AND c.deactivated IS NULL\"" | tee hosts.csv
ssh -t puppetdb-01.torproject.org "sudo -u postgres psql puppetdb -P pager=off -F',' -A -t -c \"SELECT c.certname, value_string FROM factsets fs INNER JOIN facts f ON f.factset_id = fs.id INNER JOIN fact_values fv ON fv.id = f.fact_value_id INNER JOIN fact_paths fp ON fp.id = f.fact_path_id INNER JOIN certnames c ON c.certname = fs.certname WHERE fp.name = 'virtual' AND c.deactivated IS NULL\"" | tee hosts.csv
The resulting file is a Comma-Separated Value (CSV) file which can be
used for other purposes later.
......@@ -749,7 +749,7 @@ used for other purposes later.
Possible values of the `virtual` field can be obtain with a similar
query:
ssh -t pauli.torproject.org "sudo -u postgres psql puppetdb -P pager=off -A -t -c \"SELECT DISTINCT value_string FROM factsets fs INNER JOIN facts f ON f.factset_id = fs.id INNER JOIN fact_values fv ON fv.id = f.fact_value_id INNER JOIN fact_paths fp ON fp.id = f.fact_path_id WHERE fp.name = 'virtual';\""
ssh -t puppetdb-01.torproject.org "sudo -u postgres psql puppetdb -P pager=off -A -t -c \"SELECT DISTINCT value_string FROM factsets fs INNER JOIN facts f ON f.factset_id = fs.id INNER JOIN fact_values fv ON fv.id = f.fact_value_id INNER JOIN fact_paths fp ON fp.id = f.fact_path_id WHERE fp.name = 'virtual';\""
The currently known values are: `kvm`, `physical`, and `xenu`.
......@@ -832,7 +832,7 @@ show your password on your terminal and probably in the logs as well.
Batch jobs can also be ran on all Puppet hosts with Cumin:
ssh -N -L8080:localhost:8080 pauli.torproject.org &
ssh -N -L8080:localhost:8080 puppetdb-01.torproject.org &
cumin '*' uptime
See [howto/cumin](howto/cumin) for more examples.
......@@ -1265,11 +1265,15 @@ the future if we rely more on it for deployments.
## Design
The Puppet server and PuppetDB currently live on `pauli`. That server
The Puppet master currently lives on `pauli`. That server
was setup in 2011 by weasel. It follows the configuration of the
Debian Sysadmin (DSA) Puppet server, which has its source code
available in the [dsa-puppet repository](https://salsa.debian.org/dsa-team/mirror/dsa-puppet/).
PuppetDB, which was previously hosted on `pauli`, now runs on its own dedicated
machine `puppetdb-01`. Its configuration and PostgreSQL database are managed by
the `profile::puppetdb` and `role::puppetdb` class pair.
The service is maintained by TPA and manages *all* TPA-operated
machines. Ideally, all services are managed by Puppet, but
historically, only basic services were configured through Puppet,
......@@ -1332,17 +1336,16 @@ started with the vocabulary used in this document.
* **Reports**: log of changes done on nodes recorded by the Puppet
server
* **[PuppetDB](https://puppet.com/docs/puppetdb/) server**: an application server on top of a PostgreSQL
database providing an [API](https://puppet.com/docs/puppetdb/5.2/api/index.html) to query various resources like node
database providing an [API](https://www.puppet.com/docs/puppetdb/7/api/overview) to query various resources like node
names, facts, reports and so on
### File layout
The Puppet server and PuppetDB server run on
`pauli.torproject.org`. That is where the main git repository
(`tor-puppet`) lives, in
`/srv/puppet.torproject.org/git/tor-puppet`. That repository has hooks
to populate `/etc/puppet` which is the live checkout from which the
Puppet server compiles its catalogs.
The Puppet master runs on `pauli.torproject.org`. That is where the main git
repository (`tor-puppet`) lives, in
`/srv/puppet.torproject.org/git/tor-puppet`. That repository has hooks to
populate `/etc/puppet` which is the live checkout from which the Puppet server
compiles its catalogs.
All paths below are relative to the root of that git repository.
......@@ -1728,12 +1731,16 @@ being deployed on the client side for now, and voluntarily. See the
## Logs and metrics
PuppetDB itself holds performance information about the Puppet agent
runs, which are called "reports". Those reports contain information
about changes operated on each server, how long the agent runs take
and so on. Those metrics could be made more visible by using a
dashboard, but that has not been implemented yet (see [issue
31969][]).
PuppetDB exposes a performance dashboard which is accessible via web. To reach
it, first establish an ssh forwarding to `puppetdb-01` on port 8080 as
described on this page, and point your browser at
http://localhost:8080/pdb/dashboard/index.html
PuppetDB itself also holds performance information about the Puppet agent runs,
which are called "reports". Those reports contain information about changes
operated on each server, how long the agent runs take and so on. Those metrics
could be made more visible by using a dashboard, but that has not been
implemented yet (see [issue 31969][]).
[issue 31969]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/31969
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment