diff --git a/howto/puppet.md b/howto/puppet.md index 1d6c7e716e2bc7a453bafe7c98f7093c70d82850..6463624584ce612f01bef3a80747026a2a527873 100644 --- a/howto/puppet.md +++ b/howto/puppet.md @@ -587,25 +587,91 @@ general, it's safe to use `trocla create` as it will reuse existing password. It's actually how the `trocla()` function behaves in Puppet as well. +## Exported resources + +Our Puppet configuration supports [exported resources](https://puppet.com/docs/puppet/latest/lang_exported.html), a key +component of complex Puppet deployments. Exported resources allow one +host to define a configuration that will be *exported* to the Puppet +server and then *realized* on another host. + +We commonly use this to punch holes in the firewall between nodes. For +example, this manifest in the `roles::puppetmaster` class: + + @@ferm::rule::simple { "roles::puppetmaster-${::fqdn}": + tag => 'roles::puppetmaster', + description => 'Allow Puppetmaster access to LDAP', + port => ['ldap', 'ldaps'], + saddr => $base::public_addresses, + } + +... exports a firewall rule that will, later, allow the Puppet server +to access the LDAP server (hence the `port => ['ldap', 'ldaps']` +line). This rule doesn't take effect on the host applying the +`roles::puppetmaster` class, but only on the LDAP server, through this +rather exotic syntax: + + Ferm::Rule::Simple <<| tag == 'roles::puppetmaster' |>> + +This tells the LDAP server to apply whatever rule was exported with +the `@@` syntax and the specified `tag`. Any Puppet resource can be +exported and realized that way. + ## Getting facts from other hosts -TODO: expand. - -``` -02:37:52 <bastelfreak> anarcat: query_nodes('Class[Profiles::Kafkabroker]') gets you all FQDNs from all nodes with that - class in the catalog -02:38:52 <bastelfreak> anarcat: for all ips: $ipfact = 'networking.interfaces.enp0s5.ip6' \n - query_nodes('Class[Profiles::Cephmon]', $ipfact) -02:39:24 <bastelfreak> anarcat: and something like this if you want ips from all nodes except the current ones (e.g. for - firewalling): $elknodeips = query_nodes("Class[Profiles::Elasticsearch] and ${ipfact} != '${ipv6}'", - $ipfact) -09:09:34 <bastelfreak> you can pass any factname -09:09:40 <bastelfreak> but please don't use legacy facts -09:09:44 <bastelfreak> use networking.ip -09:09:51 <bastelfreak> or networking.ip6 ! -``` - -See also `dig()`. +A common pattern in Puppet is to extract information from host A and +use it on host B. The above "exported resources" pattern can do this +for files, commands and many more resources, but sometimes we just +want a tiny bit of information to embed in a configuration file. This +could, in theory, be done with an exported [concat](https://forge.puppet.com/puppetlabs/concat) resource, but +this can become prohibitively complicated for something as simple as +an allowed IP address in a configuration file. + +For this we use the [puppetdbquery module](https://github.com/dalen/puppet-puppetdbquery), which allows us to do +elegant queries against PuppetDB. For example, this will extract the +IP addresses of all nodes with the `roles::gitlab` class applied: + + $allow_ipv4 = query_nodes('Class[roles::gitlab]', 'networking.ip') + $allow_ipv6 = query_nodes('Class[roles::gitlab]', 'networking.ip6') + +This code, in `profile::kgb_bot`, propagates those variables into a +template through the `allowed_addresses` variable, which gets expanded +like this: + + <% if $allow_addresses { -%> + <% $allow_addresses.each |String $address| { -%> + allow <%= $address %>; + <% } -%> + deny all; + <% } -%> + +This would be technically possible with a `concat` resource, but much +harder because you would need some special case when no resource is +exported (to avoid adding the `deny`) and take into account that other +configuratinos might also be needed in the file. + +Note that there's also a way to do those queries without a Forge +module, through the [Puppet query language](https://puppet.com/docs/puppetdb/5.2/api/query/tutorial-pql.html) and the +`puppetdb_query` function. The problem with that approach is that the +function is not very well documented and the query syntax is somewhat +obtuse. For example, this is what I came up with to do the equivalent +of the `query_nodes` call, above: + + $allow_ipv4 = puppetdb_query( + ['from', 'facts', + ['and', + ['=', 'name', 'networking.ip'], + ['in', 'certname', + ['extract', 'certname', + ['select_resources', + ['and', + ['=', 'type', 'Class'], + ['=', 'title', 'roles::gitlab']]]]]]]) + +It seems like I did something wrong, because that returned an empty +array. I could not figure out how to debug this, and apparently I +neded more functions (like `map` and `filter`) to get what I wanted +(see [this gist](https://gist.github.com/bastelfreak/b9620fa1892ebcc659c442b115db34f9)). I gave up at that point: the `puppetdbquery` +abstraction is much cleaner and usable. ## Revoking and generating a new certificate for a host