Skip to content
Snippets Groups Projects
Verified Commit a90d769e authored by anarcat's avatar anarcat
Browse files

start documenting ldap architecture

parent cf8d3406
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 146 additions and 17 deletions
howto/ldap.md
+ 146
− 17
View file @ a90d769e
......@@ -290,22 +290,45 @@ upload](howto/build_and_upload_debs) `userdir-ldap` as a Debian package instead.
## Pager playbook
<!-- information about common errors from the monitoring system and -->
<!-- how to deal with them. this should be easy to follow: think of -->
<!-- your future self, in a stressful situation, tired and hungry. -->
An LDAP server failure can trigger lots of emails as `ud-ldap` fails
to synchronize things. But the infrastructure should survive the
downtime, because users and passwords are *copied* over to all
hosts. In other words, authentication doesn't rely on the LDAP server
being up.
That said, if the LDAP server goes down, password changes will not
work, and the server inventory (at <https://db.torproject.org/>) will
be gone. A mitigation is to use Puppet manifests and/or PuppetDB to
get a host list and server inventory, see the [Puppet
documentation](puppet) for details.
In general, OpenLDAP is very stable and doesn't generally crash, so we
haven't had many emergencies scenarios with it yet. If anything
happens, make sure the `slapd` service is running.
The `ud-ldap` software, on the other hand, is a little more
complicated and can be hard to diagnose. TODO: expand on the failure
modes.
## Disaster recovery
<!-- what to do if all goes to hell. e.g. restore from backups? -->
<!-- rebuild from scratch? not necessarily those procedures (e.g. see -->
<!-- "Installation" below but some pointers. -->
The LDAP server is mostly built by hand and should therefore be
restored from backups in case of a catastrophic failure. Care should
be taken to keep the SSH keys of the server intact.
TODO: analyse <https://gitlab.torproject.org/tpo/tpa/team/-/issues/33908>.
# Reference
## Installation
<!-- how to setup the service from scratch -->
TODO: explain how an LDAP server could be installed from scratch.
## SLA
<!-- this describes an acceptable level of service for this service -->
The LDAP server is designed to be fault-tolerant in the sense that
it's database is copied over other hosts. It should otherwise be
highly available as it's a key component in managing users
authentication and authorization, and machines.
## Design
......@@ -315,21 +338,124 @@ the central LDAP server using the `sshdist` account, which means
things keep working when LDAP is down. Most operations can be
performed on the [db.torproject.org](https://db.torproject.org/) site or by [email](https://db.torproject.org/doc-mail.html).
### Architecture overview
This is all implemented by a tool called `ud-ldap`, inherited from the
Debian project. The project is made of a collection of bash, Python
and Perl scripts which take care of synchronizing various
configuration files to hosts based on the LDAP configuration. Most of
this section aims at documenting how this program works.
`ud-ldap` is made of two Debian packages: `userdir-ldap`, which ships
the various server- and client-side scripts (and is therefore
installed everywhere), and `userdir-ldap-cgi` which ships the web
interface (and is therefore installed only on the LDAP server).
Configuration files are generated on the server by the `ud-generate`
command, which goes over the LDAP directory and crafts a tree of
configuration files, one directory per host defined in LDAP. Then each
host pulls those configuration files with `ud-replicate`.
TODO: expand? mailgate maybe?
### Configuration file creation
An important part of `ud-ldap` is the `ud-generate`
### DNS zone file management
DNS zone files are also managed (at least partly) in LDAP. This is
automated through cron jobs, but if you're in a hurry, the zones get
generated by `ud-generate` on `alberti` (as `sshdist`?) and replicate
(?) on `nevii` with `ud-replicate` (as `root`?).
## Issues
Here is how zones are propagated from LDAP to the DNS server:
1. on the LDAP server, `ud-generate` writes various files to
`/var/cache/userdir-ldap/hosts`, one directory per host
2. on all hosts, `ud-replicate` `rsync`'s that host's directory from
the LDAP server (as the `sshdist` user) to `/var/lib/misc`
3. DNS servers (e.g. `nevii` and `falax`) are special and have a
`dns-sshfp` and `dns-zone` files
(e.g. `/var/lib/misc/thishost/dns-sshfp`) that gets generated with
all those "automatic" records from the `ipHostNumber` field in
LDAP
4. if those files change, `ud-replicate` will call
`/srv/dns.torproject.org/bin/update` as the `dnsadm` user
5. TODO: document what `bin/update` does.
4. the zone file used by bind is in
`/srv/dns.torproject.org/var/generated/torproject.org` on `nevii`,
but it doesn't include the file generated by ud-replicate, so it's
generated by something else. TODO.
TODO: document this:
> the $INCLUDE "/var/lib/misc/thishost/dns-sshfp" from the
> dns/domains.git zonefile is not parsed by bind, but by "makezonefile
> or whatever it's called to syntax check and to add the SOA header"
See also: <https://gitlab.torproject.org/tpo/tpa/team/-/issues/33766>
### Source file analysis
TODO: document those tools.
* `ud-arbimport`: Python script, ASCII text executable
* `ud-config`: Python script, ASCII text executable
* `ud-echelon`: Python script, ASCII text executable
* `ud-fingerserv`: Perl script text executable
* `ud-fingerserv2.c`: C source, ASCII text
* `ud-forwardlist`: Python script, ASCII text executable
* `ud-generate`: Python script, ASCII text executable
* `ud-gpgimport`: Python, seems unused? "Key Ring Syncronization
utility", it "maintains the key fingerprint to user ID mapping in
the directory. It takes as input a set of keyrings that represent
all keys belonging to all users in the directory. It then reads
each key and attempts to match it up to a user already in the
directory."
* `ud-gpgsigfetch`: Python script, ASCII text executable
* `ud-groupadd`: Python script, ASCII text executable
* `ud-guest-extend`: Python script, ASCII text executable
* `ud-guest-upgrade`: Python script, ASCII text executable
* `ud-homecheck`: Python script, ASCII text executable
* `ud-host`: Python script, ASCII text executable
* `ud-info`: Python script, ASCII text executable
* `ud-krb-reset`: Perl script text executable
* `UDLdap.py`: Python script, ASCII text executable
* `ud-ldapshow`: Python script, ASCII text executable
* `ud-lock`: Python script, ASCII text executable
* `ud-mailgate`: Python script, ASCII text executable
* `ud-passchk`: Python script, ASCII text executable
* `ud-replicate`: Bourne-Again shell script, ASCII text executable
* `ud-replicated`: Python script, ASCII text executable
* `ud-roleadd`: Python script, ASCII text executable
* `ud-sshlist`: Python script, ASCII text executable
* `ud-sync-accounts-to-afs`: Python script, ASCII text executable
* `ud-useradd`: Python script, ASCII text executable
* `ud-userimport`: Python script, ASCII text executable
* `ud-xearth`: Python script, ASCII text executable
* `ud-zoneupdate`: POSIX shell script, ASCII text executable
* `userdir_exceptions.py`: Python script, UTF-8 Unicode text executable
* `userdir_gpg.py`: Python script, ASCII text executable
* `userdir-ldap.conf`: Objective-C source, ASCII text
* `userdir_ldap.pth`: ASCII text
* `userdir_ldap.py`: Python script, ASCII text executable
* `userdir-ldap.schema`: ASCII text
* `userdir-ldap-slapd.conf.in`: ASCII text, with very long lines
<!-- such projects are never over. add a pointer to well-known issues -->
<!-- and show how to report problems. usually a link to the bugtracker -->
## Issues
There is no issue tracker specifically for this project, [File][] or
[search][] for issues in the [team issue tracker][search].
There is no issue tracker specifically for this project, [file][] or
[search][] for issues in the [team issue tracker][search], with the `LDAP`
label.
[File]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/new
[search]: https://gitlab.torproject.org/tpo/tpa/team/-/issues
[file]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/new
[search]: https://gitlab.torproject.org/tpo/tpa/team/-/issues?label_name%5B%5D=LDAP
## Monitoring and testing
......@@ -345,8 +471,11 @@ There is no issue tracker specifically for this project, [File][] or
## Overview
<!-- describe the overall project. should include a link to a ticket -->
<!-- that has a launch checklist -->
`ud-ldap` is decades old (the `ud-generate` manpage mentions 1999, but
it could be older) and is hard to debug and extend. This section aims
at documenting issues with the software and possible alternatives.
TODO.
## Goals
<!-- include bugs to be fixed -->
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment