... | ... | @@ -360,51 +360,107 @@ TODO: expand? mailgate maybe? |
|
|
|
|
|
TODO: diagram?
|
|
|
|
|
|
### Configuration file creation
|
|
|
### Configuration file creation and distribution
|
|
|
|
|
|
An important part of `ud-ldap` is the `ud-generate` command, which
|
|
|
generates configuration files for each host.
|
|
|
generates configuration files for each host. Then the `ud-replicate`
|
|
|
command runs on each node to `rsync` those files. Both commands are
|
|
|
ran from cron on regular intervals, the latter defined in Puppet, the
|
|
|
former hardcoded to 15 minutes.
|
|
|
|
|
|
TODO: walk through ud-generate.
|
|
|
More specifically, this is what happens:
|
|
|
|
|
|
1. on the LDAP server (currently `alberti`), `ud-generate` writes
|
|
|
various files to `/var/cache/userdir-ldap/hosts/`, one directory
|
|
|
per host
|
|
|
|
|
|
2. on all hosts, `ud-replicate` `rsync`'s that host's directory from
|
|
|
the LDAP server (as the `sshdist` user) to
|
|
|
`/var/lib/misc/$HOSTNAME` and a symlink ensures
|
|
|
`/var/lib/misc/thishost` points to that directory
|
|
|
|
|
|
TODO: walk through ud-generate. more explicitely.
|
|
|
|
|
|
### DNS zone file management
|
|
|
|
|
|
DNS zone files are also managed (at least partly) in LDAP. This is
|
|
|
automated through cron jobs, but if you're in a hurry, the zones get
|
|
|
generated by `ud-generate` on `alberti` (as `sshdist`?) and replicate
|
|
|
(?) on `nevii` with `ud-replicate` (as `root`?).
|
|
|
One of the configuration files `ud-generate` generates are,
|
|
|
critically, the `dns-sshfp` and `dns-zone` files.
|
|
|
|
|
|
The `dns-sshfp` file holds the following records mapped to LDAP
|
|
|
`host` fields:
|
|
|
|
|
|
| DNS record | LDAP host field | Notes |
|
|
|
| ---------- | --------------- | ----- |
|
|
|
| `SSHFP` | `sshRSAHostKey` | extra entries possible with the `sshfphostname` field |
|
|
|
| `A`, `AAAA` | `ipHostNumber` | TTL overridable with the `dnsTTL` field |
|
|
|
| `HINFO` | `architecture` and `machine` | |
|
|
|
| `MX` | `mXRecord` | |
|
|
|
|
|
|
The `dns-zone` file contains *user*-specific DNS entries. If a `user`
|
|
|
object has a `dnsZoneEntry` field, that entry is written to the file
|
|
|
directly. A `TXT` record with the user's email address and their PGP
|
|
|
key fingerprint is also added for identification. That file is not in
|
|
|
use in TPO at the moment, but is (probably?) the mechanism behind the
|
|
|
user-editable `debian.net` zone.
|
|
|
|
|
|
Those files only get *distributed* to DNS servers (e.g. `nevii` and
|
|
|
`falax`), which are marked with the `DNS` flag in the `exportOptions`
|
|
|
field in LDAP.
|
|
|
|
|
|
Here is how zones are propagated from LDAP to the DNS server:
|
|
|
|
|
|
1. on the LDAP server, `ud-generate` writes various files to
|
|
|
`/var/cache/userdir-ldap/hosts`, one directory per host
|
|
|
1. `ud-replicate` will pull the files with `rsync`, as explained in
|
|
|
the previous section
|
|
|
|
|
|
2. on all hosts, `ud-replicate` `rsync`'s that host's directory from
|
|
|
the LDAP server (as the `sshdist` user) to `/var/lib/misc`
|
|
|
2. if the `dns-zone` or `dns-sshfp` files change, `ud-replicate` will
|
|
|
call `/srv/dns.torproject.org/bin/update` (from `dns_helpers.git`)
|
|
|
as the `dnsadm` user, which creates the final zonefile in
|
|
|
`/srv/dns.torproject.org/var/generated/torproject.org`
|
|
|
|
|
|
The `bin/update` script does the following:
|
|
|
|
|
|
1. pulls the `auto-dns.git` and `domains.git` git repositories
|
|
|
|
|
|
2. updates the DNSSEC keys (with `bin/update-keys`)
|
|
|
|
|
|
3. update the GeoIP distribution mechanism (with `bin/update-geo`)
|
|
|
|
|
|
4. builds the service includes from the `auto-dns` directory (with
|
|
|
`auto-dns/build-services`), which writes the
|
|
|
`/srv/dns.torproject.org/var/services-auto/all` file
|
|
|
|
|
|
3. DNS servers (e.g. `nevii` and `falax`) are special and have a
|
|
|
`dns-sshfp` and `dns-zone` files
|
|
|
(e.g. `/var/lib/misc/thishost/dns-sshfp`) that gets generated with
|
|
|
all those "automatic" records from the `ipHostNumber` field in
|
|
|
LDAP
|
|
|
5. for each domain in `domains.git`, calls `write_zonefile` (from
|
|
|
`dns_helpers.git`), which in turn:
|
|
|
|
|
|
4. if those files change, `ud-replicate` will call
|
|
|
`/srv/dns.torproject.org/bin/update` as the `dnsadm` user
|
|
|
1. increments the serial number in the `.serial` state file
|
|
|
2. generate a zone header with the new serial number
|
|
|
3. include the zone from `domains.git`
|
|
|
4. compile it with [named-compilezone(8)][], which is the part
|
|
|
that expands the various `$INCLUDE` directives
|
|
|
|
|
|
5. TODO: document what `bin/update` does.
|
|
|
6. then calls `dns-update` (from `dns_helpers.git`) which rewrites
|
|
|
the `named.conf` snippet and reloads bind, if needed
|
|
|
|
|
|
4. the zone file used by bind is in
|
|
|
`/srv/dns.torproject.org/var/generated/torproject.org` on `nevii`,
|
|
|
but it doesn't include the file generated by ud-replicate, so it's
|
|
|
generated by something else. TODO.
|
|
|
[named-compilezone(8)]: https://manpages.debian.org/named-compilezone.8
|
|
|
|
|
|
TODO: document this:
|
|
|
The various `$INCLUDE` directives in the `torproject.org` zonefile are
|
|
|
currently:
|
|
|
|
|
|
> the $INCLUDE "/var/lib/misc/thishost/dns-sshfp" from the
|
|
|
> dns/domains.git zonefile is not parsed by bind, but by "makezonefile
|
|
|
> or whatever it's called to syntax check and to add the SOA header"
|
|
|
* `/var/lib/misc/thishost/dns-sshfp` - generated on the LDAP server
|
|
|
by `ud-generate`, contains SSHFP records for each host
|
|
|
* `/srv/dns.torproject.org/puppet-extra/include-torproject.org`:
|
|
|
generated by Puppet modules which call the `dnsextras` module. This
|
|
|
is used, among other things, for TLSA records for HTTPS and SMTP
|
|
|
services
|
|
|
* `/srv/dns.torproject.org/var/services-auto/all`: generated by the
|
|
|
`build-services` script in the `auto-dns.git` directory
|
|
|
* `/srv/letsencrypt.torproject.org/var/hook/snippet`: generated by
|
|
|
the `bin/le-hook` in the `letsencrypt-domains.git` repository, to
|
|
|
authenticate against Let's Encrypt and generate [TLS](tls)
|
|
|
certificates.
|
|
|
|
|
|
See also: <https://gitlab.torproject.org/tpo/tpa/team/-/issues/33766>
|
|
|
Note that this procedure fails when the git server is unavailable, see
|
|
|
[issue 33766](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33766) for details.
|
|
|
|
|
|
### Source file analysis
|
|
|
|
... | ... | |