Skip to content

Changes

Page history
move all services from howto to service/ authored by anarcat's avatar anarcat 
This creates redirects for all moved pages, removes them from the
howto page, and deletes a couple of dead pages that were just
redirects.

It does *not* go through all internal links to fix them, but we do
take care of the service.md index as well.
Show whitespace changes
Inline Side-by-side
service/ldap.md 0 → 100644
View page @ acc169fd
[LDAP](https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol) is a directory service we use to inventory the users,
groups, passwords, (some) email forwards and machines. It distributes
some configuration and password files to all machines and can reload
services.
Note that this documentation needs work, particularly regarding user
management procedures, see [issue 40129](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40129).
[[_TOC_]]
# Tutorial
Our LDAP configuration is rather exotic. You will typically use the
[web interface][db.torproject.org] and the OpenPGP-enabled email interface. This
documentation aims at getting you familiar with the basics.
[db.torproject.org]: https://db.torproject.org/
## Getting to know LDAP
You should have received an email like this when your LDAP account was
created:
Subject: New ud-ldap account for <your name here>
That includes information about how to configure email forwarding and
SSH keys. You should follow those steps to configure your SSH key to
get SSH access to servers (see [ssh-jump-host](doc/ssh-jump-host/)).
## How to change my email forward?
If you use Thunderbird and use it to manage your OpenPGP key, compose a new
plain text (not HTML) message to `changes@db.torproject.org`, enter any subject
line and write this in the message body:
emailForward: user@example.com
Before sending the email, open the OpenPGP drop-down menu at the top of the
compose window and click `Digitally Sign`.
If you use GnuPG, send an (inline!) signed OpenPGP email to
`changes@db.torproject.org` to change your email forward.
A command like this, in a UNIX shell, would do it:
echo "emailForward: user@example.com" | gpg --armor --sign
Then copy-paste that in your email client, making sure to avoid
double-signing the email and sending in clear text (instead of HTML).
The email forward can also be changed in the [web interface][db.torproject.org].
## Password reset
If you have lost or forgotten your LDAP password or if you are are newly hired
by TPI (congratulations!) and don't know your password yet, you can have it
reset by sending a PGP signed message to the mail gateway.
The email should:
* be sent to `chpasswd@db.torproject.org`
* be composed in plain text (not HTML)
* be PGP signed by your key
* have exactly (and just) this text as the message body: `Please change my Tor
password`
If you use Thunderbird and use it to manage your OpenPGP key, compose a new
message in plain text (not HTML). You can configure sending emails in plaintext
in your account settings, or if your new messages are usually composed in HTML
you can hold the Shift key while clicking on the "+ New Message" button. Enter
any subject line and write the message body described above.
Before sending the email, open the OpenPGP drop-down menu at the top of the
compose window and click `Digitally Sign`.
_Or_, you can use GnuPG directly and then send an (inline!) email with your
client of choice. A command like the following, in a UNIX shell, will create the
signed text that you can copy-paste in your email. Make sure to avoid
double-signing the email and sending it in clear text (instead of HTML):
echo "Please change my Tor password" | gpg --armor --sign
However you sent your signed email, the daemon will then respond with a new
randomized password encrypted with your key. You can then use the [update
form](https://db.torproject.org/login.html) with your new password to change
your it to a strong password, in the "Change password" field, that you can
remember or (preferably) a stronger password (longer and more random) stored in
your password manager. Note: on that "update form" login page the button you
should use to login is, unintuitively, labeled "Update my info"
You cannot set a new password via the mail gateway.
Alternatively, you can do without a password and use PGP to manipulate
your LDAP information through the [mail gateway](#operate-the-mail-gateway), which includes
instructions on SSH public key authentication, for example.
## How do I update my OpenPGP key?
LDAP requires an OpenPGP key fingerprint in its records and uses that
trust anchor to review changes like [resetting your password](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#password-reset) or
[uploading an SSH key](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/howto/ldap#uploading-a-ssh-user-key).
You can't, unfortunately, update the OpenPGP key yourself. Setting the
key should have been done as part of your on-boarding. If it has not
been done or you need to perform changes on the key, you should [file
an issue with TPA](https://gitlab.torproject.org/tpo/tpa/team/-/issues/new), detailing what change you want. Include a copy
of the public key certificate.
To check whether your fingerprint is already stored in LDAP, search
for your database entry in <https://db.torproject.org/search.cgi> and
check the "PGP/GPG fingerprint" field.
We acknowledge this workflow is far from ideal, see
[tpo/tpa/team#40129](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40129) and [tpo/tpa/team#29671](https://gitlab.torproject.org/tpo/tpa/team/-/issues/29671) for further
discussion and future work.
# How-to
## Set a sudo password
See the [sudo password user configuration](doc/accounts#host-specific-passwords--sudo-passwords).
## Operate the mail gateway
<!-- warning: this documentation is copied from `html/doc-mail.wml` -->
<!-- in the userdir-ldap-cgi.git repository. It should be updated in -->
<!-- both places if changes are made. -->
The LDAP directory has a PGP secured mail gateway that allows users to safely
and conveniently effect changes to their entries. It makes use of PGP signed
input messages to positively identify the user and to confirm the validity of
the request. Furthermore it implements a replay cache that prevents the gateway
from accepting the same message more than once.
There are three functions logically split into 3 separate email addresses that
are implemented by the gateway: ping, new password and changes. The function to
act on is the first argument to the program.
Error handling is currently done by generating a bounce message and passing
descriptive error text to the mailer. This can generate a somewhat hard to read
error message, but it does have all the relevant information.
### ping
The ping command simply returns the users public record. It is useful for
testing the gateway and for the requester to get a basic dump of their record.
In future this address might 'freshen' the record to indicate the user is
alive. Any PGP signed message will produce a reply.
### New Password
If a user loses their password they can request that a new one be generated for
them. This is done by sending the phrase "Please change my Tor password" to
`chpasswd@db.torproject.org`. The phrase is required to prevent the daemon from
triggering on arbitrary signed email. The best way to invoke this feature is
with:
echo "Please change my Tor password" | gpg --armor --sign | mail chpasswd@db.torproject.org
After validating the request the daemon will generate a new random password,
set it in the directory and respond with an encrypted message containing the
new password. The password can be changed using one of the other interface
methods.
### Changes
An address (`changes@db.torproject.org`) is provided for making almost arbitrary
changes to the contents of the record. The daemon parses its input line by line
and acts on each line in a command oriented manner. Anything, except for
passwords, can be changed using this mechanism. Note however that because this
is a mail gateway it does stringent checking on its input. The other tools
allow fields to be set to virtually anything, the gateway requires specific
field formats to be met.
* **field**: A line of the form `field: value` will change the
contents of the field to value. Some simple checks are performed on
value to make sure that it is not set to nonsense. You can't set an
empty string as value, use `del` instead (see below). The values that
can be changed are: `loginShell`, `emailForward`, `ircNick`, `jabberJID`,
`labledURI`, and `VoIP`
* **del field**: A line of the form `del field` will completely remove all
occurrences of a field. Useful e.g. to unset your vacation status.
* **SSH keys changes**, see [uploading a SSH user key](#uploading-a-ssh-user-key)
* **show**: If the single word show appears on a line in a PGP signed
mail then a PGP encrypted version of the entire record will be
attached to the resulting email. For example:
echo show | gpg --clearsign | mail changes@db.torproject.org
Note that the changes alias does not handle PGP/MIME emails.
After processing the requests the daemon will generate a report which
contains each input command and the action taken. If there are any
parsing errors processing stops immediately, but valid changes up to
that point are processed.
### Notes
In this document PGP refers to any message or key that GnuPG is able to
generate or parse, specifically it includes both PGP2.x and OpenPGP (aka GnuPG)
keys.
Due to the replay cache the clock on the computer that generates the signatures
has to be accurate to at least one day. If it is off by several months or more
then the daemon will outright reject all messages.
## Uploading a SSH user key
<!-- warning: this documentation is copied from a part of `html/doc-mail.wml` -->
<!-- in the userdir-ldap-cgi.git repository. It should be updated in -->
<!-- both places if changes are made. -->
To upload a key into your `authorized_keys` file on all servers,
simply place the key on a line by itself, sign the message and send it
to `changes@db.torproject.org`. The full SSH key format specification
is supported, see sshd(8). Probably the most common way to use this
function will be
gpg --armor --sign < ~/.ssh/id_rsa.pub | mail changes@db.torproject.org
Which will set your `authorized_keys` to `~/.ssh/id_rsa.pub` on all
servers.
Supported key types are RSA (at least 2048 bits) and Ed25519.
Multiple keys per user are supported, but they must all be sent at
once. To retrieve the existing SSH keys in order to merge existing
keys with new ones, use the `show` command documented above.
Keys can be exported to a subset of machines by prepending
`allowed_hosts=$fqdn,$fqdn2` to the specific key. The allowed machines
must only be separated by a comma. Example:
allowed_hosts=ravel.debian.org,gluck.debian.org ssh-rsa AAAAB3Nz..mOX/JQ== user@machine
ssh-rsa AAAAB3Nz..uD0khQ== user@machine
## SSH host keys verification
<!-- warning: this documentation is copied from `html/doc-hosts.wml` -->
<!-- in the userdir-ldap-cgi.git repository. It should be updated in -->
<!-- both places if changes are made. -->
The SSH host keys are stored in the LDAP database. The key and its
fingerprint will be displayed alongside machine details in the
[machine list](https://db.torproject.org/machines.cgi).
Developers that have a secure path to a DNSSEC enabled resolver can
verify the existing SSHFP records by adding `VerifyHostKeyDNS yes` to
their `~/.ssh/config` file.
On machines in which are updated from the LDAP database,
`/etc/ssh/ssh_known_hosts` contains the keys for all hosts in this
domain.
Developers should add `StrictHostKeyChecking yes` to their
`~/.ssh/config` file so that they only connect to trusted
hosts. Either with the DNSSEC records or the file mentioned above,
nearly all hosts in the domain can be trusted automatically.
Developers can also execute `ud-host -f` or `ud-host -f -h host` on a
server in order to display all host fingerprints or only the
fingerprints of a particular host in order to compare it with the
output of ssh on an external host.
## Know when will my change take effect?
Once a change is saved to LDAP, the actual change will take at least 5
minutes and at most 15 minutes to propagate to the relevant host. See
the [configuration file distribution
section](#configuration-file-distribution) for more details on why it
is so.
## Locking an account
See [the user retirement procedures](howto/retire-a-user).
## Connecting to LDAP
LDAP is not accessible to the outside world, so you need to get behind
the firewall. Most operations are done directly on the LDAP server, by
logging in as a regular user on `db.torproject.org` (currently
`alberti`).
Once that's resolved, you can use [ldapvi(1)](https://manpages.debian.org/ldapvi.1.en.html) or [ldapsearch(1)](https://manpages.debian.org/ldapsearch.1.en.html)
to inspect the database. User documentation on that process is in
[doc/accounts](doc/accounts) and <https://db.torproject.org>. See also the rest
of this documentation.
## Restoring from backups
There's no special backup procedures for the LDAP server: it's backed
up like everything else in the [backup](service/backup) system.
To restore the OpenLDAP database, you need to head over the Bacula
director, and enter the console:
ssh -tt bacula-director-01 bconsole
Then call the `restore` command and select `6: Select backup for a
client before a specified time.` Then pick the server (currently
`alberti.torproject.org`) and a date. Then you need to "mark" the
right files:
cd /var/lib/ldap
mark *
done
Then confirm the restore. The files will end up in
`/var/tmp/bacula-restores` on the LDAP server.
The next step depends on whether this is a partial or total
restore.
### Partial restore
If you only need to access a specific field or user or part of the
database, you can use `slapcat` to dump the database from the restored
files even if the server is not running. You first need to "configure"
a "fake" server in the restore directory. You will need to create two
files under `/var/tmp/bacula-restores`:
* `/var/tmp/bacula-restores/etc/ldap/slapd.conf`
* `/var/tmp/bacula-restores/etc/ldap/userdir-ldap-slapd.conf`
They can be copied from `/etc`, with the following modifications:
diff -ru /etc/ldap/slapd.conf etc/ldap/slapd.conf
--- /etc/ldap/slapd.conf 2011-10-30 15:43:43.000000000 +0000
+++ etc/ldap/slapd.conf 2019-11-25 19:48:57.106055596 +0000
@@ -17,10 +17,10 @@
# Where the pid file is put. The init.d script
# will not stop the server if you change this.
-pidfile /var/run/slapd/slapd.pid
+pidfile /var/tmp/bacula-restores/var/run/slapd/slapd.pid
# List of arguments that were passed to the server
-argsfile /var/run/slapd/slapd.args
+argsfile /var/tmp/bacula-restores/var/run/slapd/slapd.args
# Read slapd.conf(5) for possible values
loglevel none
@@ -57,4 +57,4 @@
#backend <other>
# userdir-ldap
-include /etc/ldap/userdir-ldap-slapd.conf
+include /var/tmp/bacula-restores/etc/ldap/userdir-ldap-slapd.conf
diff -ru /etc/ldap/userdir-ldap-slapd.conf etc/ldap/userdir-ldap-slapd.conf
--- /etc/ldap/userdir-ldap-slapd.conf 2019-11-13 20:55:58.789411014 +0000
+++ etc/ldap/userdir-ldap-slapd.conf 2019-11-25 19:49:45.154197081 +0000
@@ -5,7 +5,7 @@
suffix "dc=torproject,dc=org"
# Where the database file are physically stored
-directory "/var/lib/ldap"
+directory "/var/tmp/bacula-restores/var/lib/ldap"
moduleload accesslog
overlay accesslog
@@ -123,7 +123,7 @@
database hdb
-directory "/var/lib/ldap-log"
+directory "/var/tmp/bacula-restores/var/lib/ldap-log"
suffix cn=log
#
sizelimit 10000
Then `slapcat` is able to read those files directly:
slapcat -f /var/tmp/bacula-restores/etc/ldap/slapd.conf -F /var/tmp/bacula-restores/etc/ldap
Copy-paste the stuff you need into `ldapvi`.
### Full rollback
Untested procedure.
If you need to roll back the *entire* server to this version, you
first need to stop the LDAP server:
service slapd stop
Then move the files into place (in `/var/lib/ldap`):
mv /var/lib/ldap{,.orig}
cp -R /var/tmp/bacula-restores/var/lib/ldap /var/lib/ldap
chown -R openldap:openldap /var/lib/ldap
And start the server again:
service slapd start
## User management
### Listing members of a group
To tell which users are part of a given group (LDAP or otherwise), you
can use the [getent(1)](https://manpages.debian.org/getent.1) command. For example, to see which users
are part of the `tordnsel` group, you would call this command:
$ getent group tordnsel
tordnsel:x:1532:arlo,arma
In the above, `arlo` and `arma` are members of the `tordnsel` group.
The fields in the output are in the format of the [group(5)](https://manpages.debian.org/buster/manpages/group.5.en.html) file.
Note that the group membership will vary according to the machine on
which the command is run, as not all users are present everywhere.
### Creating users
Users can be created for either individuals or servers (role account). Refer to
the sections [Creating a new user](howto/create-a-new-user#creating-a-new-user)
and [Creating a role](howto/create-a-new-user#creating-a-role) of the page
about creating a new user for procedures to create users of both types.
### Adding/removing users in a group
Using this magical `ldapvi` command on the LDAP server
(`db.torproject.org`):
ldapvi -ZZ --encoding=ASCII --ldap-conf -h db.torproject.org -D "uid=$USER,ou=users,dc=torproject,dc=org"
... you get thrown in a text editor showing you the entire dump of the
LDAP database. Be careful.
To add or remove a user to/from a group, first locate that user with
your editor search function (e.g. in `vi`, you'd type
<kbd>/uid=ahf</kbd> to look for the `ahf` user). You should see a
block that looks like this:
351 uid=ahf,ou=users,dc=torproject,dc=org
uid: ahf
objectClass: top
objectClass: inetOrgPerson
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianDeveloper
uidNumber: 2103
gidNumber: 2103
[...]
supplementaryGid: torproject
To add or remove a group, simply add or remove a `supplementaryGid`
line. For example, in the above, we just added this line:
supplementaryGid: tordnsel
to add `ahf` to the `tordnsel` group.
Save the file and exit the editor. `ldapvi` will prompt you to confirm
the changes, you can review with the <kbd>v</kbd> key or save with
<kbd>y</kbd>.
### Adding/removing an admin
The LDAP administrator group is a special group that is not defined
through the `supplementaryGid` field, but by adding users into the
group itself. With `ldapvi` (see above), you need to add a `member:`
line, for example:
```
2 cn=LDAP Administrator,ou=users,dc=torproject,dc=org
objectClass: top
objectClass: groupOfNames
cn: LDAP administrator
member: uid=anarcat,ou=users,dc=torproject,dc=org
```
To remove the user from the admin group, remove the line.
The group grants the user access to administer LDAP directly, for
example making any change through `ldapvi`.
Typically, admins will also be part of the `adm` group, with a normal
line:
supplementaryGid: adm
## Searching LDAP
This will load a text editor with a dump of all the users (useful to
modify an existing user or add a new one):
ldapvi -ZZ --encoding=ASCII --ldap-conf -h db.torproject.org -D "uid=$USER,ou=users,dc=torproject,dc=org"
This dump all known hosts in LDAP:
ldapsearch -ZZ -Lx -H ldap://db.torproject.org -b "ou=hosts,dc=torproject,dc=org"
Note that this will only work on the LDAP host itself or on
whitelisted hosts which are few right now. Also note that this uses an
"anonymous" connection, which means that some (secret) fields might
not show up. For hosts, that's fine, but if you search for users, you
will need to use authentication. This, for example, will dump all
users with an SSH key:
ldapsearch -ZZ -LxW -H ldap://db.torproject.org -D "uid=$USER,ou=users,dc=torproject,dc=org" -b "ou=users,dc=torproject,dc=org" '(sshRSAAuthKey=*)'
Note how we added a [search filter](https://ldap.com/ldap-filters/) (`(sshRSAAuthKey=*)`) here. We could
also have parsed the output in a script or bash, but this can actually
be much simpler. Also note that the previous searches dump the entire
objects. Sometimes it might be useful to only *list* the object
handles or certain fields. For example, this will list all hosts
`rebootPolicy` attribute:
ldapsearch -H ldap://db.torproject.org -x -ZZ -b ou=hosts,dc=torproject,dc=org -LLL '(objectClass=*)' 'rebootPolicy'
This will list all servers with a manual reboot policy:
ldapsearch -H ldap://db.torproject.org -x -ZZ -b ou=hosts,dc=torproject,dc=org -LLL '(rebootPolicy=manual)' ''
Note here the empty (`''`) attribute list.
To list hosts that do *not* have a reboot policy, you need a boolean modifier:
ldapsearch -H ldap://db.torproject.org -x -ZZ -b ou=hosts,dc=torproject,dc=org -LLL '(!(rebootPolicy=manual))' ''
Such filters can be stacked to do complex searches. For example, this
filter lists all active accounts:
ldapsearch -ZZ -vLxW -H ldap://db.torproject.org -D "uid=$USER,ou=users,dc=torproject,dc=org" -b "ou=users,dc=torproject,dc=org" '(&(!(|(objectclass=debianRoleAccount)(objectClass=debianGroup)(objectClass=simpleSecurityObject)(shadowExpire=1)))(objectClass=debianAccount))'
This lists users with access to Gitolite:
((allowedGroups=git-tor)|(exportOptions=GITOLITE))
... inactive users:
(&(shadowExpire=1)(objectClass=debianAccount))
## Modifying the schema
If you need to add, change or remove a field in the *schema* of the
LDAP database, it is a different, and complex operation. You will only
need to do this if you launch a new service that (say) requires a new
password specifically for that service.
The schema is maintained in the [userdir-ldap.git](https://gitlab.torproject.org/tpo/tpa/userdir-ldap/) repository. It
is stored in the `userdir-ldap.schema` file. Assuming the modified
object is a `user`, you would need to edit the file in three places:
1. as a comment, in the beginning, to allocate a new field, for
example:
@@ -113,6 +113,7 @@
# .45 - rebootPolicy
# .46 - totpSeed
# .47 - sshfpHostname
+# .48 - mailPassword
#
# .3 - experimental LDAP objectClasses
# .1 - debianDeveloper
This is purely informative, but it is important as it serves as a
central allocation point for that numbering system. Also note that
the entire schema lives under a branch of the [Debian.org IANA OID
allocation](https://dsa.debian.org/iana/). If you reuse the OID
space of Debian, it's important to submit the change to Debian
sysadmins (`dsa@debian.org`) so they merge your change and avoid
clashes.
2. create the actual attribute, somewhere next to a similar attribute
or after the previous OID, in this case we created an attributed
called `mailPassword` right after `rtcPassword`, since other
passwords were also grouped there:
attributetype ( 1.3.6.1.4.1.9586.100.4.2.48
NAME 'mailPassword'
DESC 'mail password for SMTP'
EQUALITY octetStringMatch
SYNTAX 1.3.6.1.4.1.1466.115.121.1.40 )
3. finally, the new attribute needs to be added to the
`objectclass`. in our example, the field was added alongside the
other password fields in the `debianAccount` `objectclass`, which
looked like this after the change:
objectclass ( 1.3.6.1.4.1.9586.100.4.1.1
NAME 'debianAccount'
DESC 'Abstraction of an account with POSIX attributes and UTF8 support'
SUP top AUXILIARY
MUST ( cn $ uid $ uidNumber $ gidNumber )
MAY ( userPassword $ loginShell $ gecos $ homeDirectory $ description $ mailDisableMessage $ sudoPassword $ webPassword $ rtcPassword $ mailPassword $ totpSeed ) )
Once that schema file is propagated to the LDAP server, this should
automatically be loaded by `slapd` when it is restarted (see
below). But the ACL for that field should also be modified. In our
case, we had to add the `mailPassword` field to two ACLs:
--- a/userdir-ldap-slapd.conf.in
+++ b/userdir-ldap-slapd.conf.in
@@ -54,7 +54,7 @@ access to attrs=privateSub
by * break
# allow users write access to an explicit subset of their fields
-access to attrs=c,l,loginShell,ircNick,labeledURI,icqUIN,jabberJID,onVacation,birthDate,mailDisableMessage,gender,emailforward,mailCallout,mailGreylisting,mailRBL,mailRHSBL,mailWhitelist,mailContentInspectionAction,mailDefaultOptions,facsimileTelephoneNumber,telephoneNumber,postalAddress,postalCode,loginShell,onVacation,latitude,longitude,VoIP,userPassword,sudoPassword,webPassword,rtcPassword,bATVToken
+access to attrs=c,l,loginShell,ircNick,labeledURI,icqUIN,jabberJID,onVacation,birthDate,mailDisableMessage,gender,emailforward,mailCallout,mailGreylisting,mailRBL,mailRHSBL,mailWhitelist,mailContentInspectionAction,mailDefaultOptions,facsimileTelephoneNumber,telephoneNumber,postalAddress,postalCode,loginShell,onVacation,latitude,longitude,VoIP,userPassword,sudoPassword,webPassword,rtcPassword,mailPassword,bATVToken
by self write
by * break
@@ -64,7 +64,7 @@ access to attrs=c,l,loginShell,ircNick,labeledURI,icqUIN,jabberJID,onVacation,bi
##
# allow authn/z by anyone
-access to attrs=userPassword,sudoPassword,webPassword,rtcPassword,bATVToken
+access to attrs=userPassword,sudoPassword,webPassword,rtcPassword,mailPassword,bATVToken
by * compare
# readable only by self
If those are the only required changes, it is acceptable to directly
make those changes directly on the LDAP server, as long as the *exact*
same changes are performed in the git repository.
It is preferable, however, to [build and
upload](howto/build_and_upload_debs) `userdir-ldap` as a Debian package instead.
## Deploying new userdir-ldap releases
Our userdir-ldap codebase is deployed through Debian packages built by
hand on TPA's members computers, from our [userdir-ldap
repository][]. Typically, when we make changes to that repository, we
should make sure we send the patches upstream, to the [DSA
userdir-ldap repository][]. The right way to do *that* is to send the
patch by email, to <mailto:dsa@debian.org>, since they do not have
merge requests enabled on that repository.
If you are lucky, we will have the latest version of the upstream code
and your patch will apply cleanly upstream. If unlucky, you'll
actually need to merge with upstream first. This process is generally
done through those steps:
1. `git merge` the upstream changes, and resolve the conflicts
2. update the changelog (make sure you have the upstream version with
`~tpo1` as a suffix so that upgrades work when if we ever catch up
with upstream)
3. build the Debian package: `git buildpackage`
4. deploy the Debian package
Note that you may want to review our feature branches to see if
our changes have been accepted upstream and, if not, update and resend
the feature branches. See the [branch policy](#branching-policy) documentation for
more ideas.
Note that unless the change is trivial, the Debian package should be
deployed *very* carefully. Because userdir-ldap is such a critical
piece of infrastructure, it can easily break stuff like PAM and
logins, so it is important to deploy it one machine at a time, and run
`ud-replicate` on the deployed machine (and `ud-generate` if the
machine is the LDAP server).
So "deploy the Debian package" should actually be done by copying, by
hand, the package to specific servers over SSH, and only after testing
there, [uploading it to the Debian archive](howto/build_and_upload_debs).
Note that it's probably a good idea to update the [userdir-ldap-cgi
repository][] alongside userdir-ldap. The above process should
similarly apply.
## Pager playbook
An LDAP server failure can trigger lots of emails as `ud-ldap` fails
to synchronize things. But the infrastructure should survive the
downtime, because users and passwords are *copied* over to all
hosts. In other words, authentication doesn't rely on the LDAP server
being up.
In general, OpenLDAP is very stable and doesn't generally crash, so we
haven't had many emergencies scenarios with it yet. If anything
happens, make sure the `slapd` service is running.
The `ud-ldap` software, on the other hand, is a little more
complicated and can be hard to diagnose. It has a large number of
moving parts (Python, Perl, Bash, Shell scripts) and talks over a
large number of protocols (email, DNS, HTTPS, SSH, finger). The
failure modes documented here are far from exhaustive and you should
expect exotic failures and error messages.
### LDAP server failure
That said, if the LDAP server goes down, password changes will not
work, and the server inventory (at <https://db.torproject.org/>) will
be gone. A mitigation is to use Puppet manifests and/or PuppetDB to
get a host list and server inventory, see the [Puppet
documentation](howto/puppet) for details.
### Git server failure
The LDAP server will fail to regenerate (and therefore update) zone
files and zone records if the Git server is unavailable. This is
described in [issue 33766](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33766). The fix is to recover the git server. A
workaround is to run this command on the primary DNS server (currently
`nevii`):
sudo -u dnsadm /srv/dns.torproject.org/bin/update --force
### Deadlocks in ud-replicate
The `ud-replicate` process keeps a "reader" lock on the LDAP
server. If for some reason the network transport fails, that lock
might be held on forever. This happened in the past on hosts with
flaky network or ipsec problems that null-routed packets between ipsec
nodes.
There is a Prometheus metric that will detect stale synchronization.
The fix is to find the offending locked process and kill it. In
desperation:
pkill -u sshdist rsync
... but really, you should carefully review the rsync processes before
killing them all like that. And obviously, fixing the underlying
network issue would be important to avoid such problems in the future.
Also note that the lock file is in
`/var/cache/userdir-ldap/hosts/ud-generate.lock`, and `ud-generate`
tries to get a *write* lock on the file. This implies that a deadlock
will also affect file generation and keep `ud-generate` from
generating fresh config files.
Finally, `ud-replicate` also holds a lock on `/var/lib/misc` on the
*client* side, but that rarely causes problems.
### Troubleshooting changes@ failures
A common user question is that they are unable to change their SSH
key. This can happen if their email client somehow has trouble sending
a PGP signature correctly. Most often than not, this is because their
email client does a line wrap or somehow corrupts the OpenPGP
signature in the email.
A good place to start looking for such problems is the log files on
the LDAP server (currently `alberti`). For example, this has a trace
of all the emails received by the `changes@` alias:
/srv/db.torproject.org/mail-logs/received.changes
A common problem is people using `--clearsign` instead of `--sign`
when sending an SSH key. When that happens, many email clients
(including Gmail) will word-wrap the SSH key after the comment,
breaking the signature. For example, this might happen:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKxqYYEeus8dRXBHhLsp0SjH7ut2X8UM9hdXN=
wJIl89otcJ5qKoXj90K9hq8eBjG2KuAZtp0taGQHqzBOFK+sFm9/gIqvzzQ07Pn0xtkmg10Hunq=
vPKMj4gDFLIqTF0WSPA2E6L/TWaeVJ+IiGuE49j+0Ohd7UFDEquM1H/zno22vIEm/dxWLPWD9gG=
MmwBghvfK/dRyzSEDGlAVeWLzoIvVOG12/ANgic3TlftbhiLKTs52hy8Qhq/aQBqd0McaE4JGxe=
9k71OCg+0WHVS4q7HVdTUqT3VFFfz0kjDzYTYQQcHMqPHvYzZghxMVCmteNdJNwJmGSNPVaUeJG=
MumJ9
anarcat@curie
-----BEGIN PGP SIGNATURE-----
[...]
-----END PGP SIGNATURE-----
Using `--sign --armor` will work around this problem, as the original
message will all be ASCII-armored.
### Dependency loop on new installs
Installing a new server requires granting the new server access
various machines, including [puppet](howto/puppet) and the LDAP server
itself. This is granted ... by Puppet through LDAP!
So a server cannot register itself on the LDAP server and needs an
operator to first create a `host` snippet on the LDAP server, and then
run Puppet on the Puppet server. This is documented in the
[installation notes](howto/new-machine).
### Server certificate renewal
The LDAP server uses a self-signed CA certificate that clients use to
verify TLS connections, both on port 389 (via STARTTLS) and port 636.
When the `db.torproject.org.pem` certificate nears its expiration
date, Prometheus will spawn warnings.
To renew this certificate, log on to `alberti.torproject.org` and create a text
file named `db.torproject.org.cfg` with this content:
ca
signing_key
encryption_key
expiration_days = 730
cn = db.torproject.org
Then the new certificate can be generated using `certtool`:
certtool --generate-self-signed \
--load-privkey /etc/ldap/db.torproject.org.key \
--outfile db.torproject.org.pem \
--template db.torproject.org.cfg
Copy the contents of the certificate on your machine:
cat db.torproject.org.pem
To bootstrap the new certificate, follow these steps first on `alberti`:
puppet agent --disable "updating LDAP certificate"
cp db.torproject.org.pem /etc/ssl/certs/db.torproject.org.pem
systemctl restart slapd.service
You can then verify OpenLDAP is working correctly by running:
ldapsearch -n -v -ZZ -x -H ldap://db.torproject.org
If it works, the process can be continued by deploying the certificate
manually on `pauli` (the Puppet server):
puppet agent --disable "updating LDAP certificate"
# replace the old certificate manually
cat > /etc/ssl/certs/db.torproject.org.pem <<EOF
-----BEGIN CERTIFICATE-----
[...]
-----END CERTIFICATE-----
EOF
# fully restart Puppet
systemctl stop apache2
systemctl start apache2
At this point, the new certificate can be replaced on the `tor-puppet`
repository, in `modules/ldap_client_config/files/db.torproject.org.pem`.
Lastly, run `puppet agent --enable` on `alberti` and `pauli` and trigger a
Puppet run on all nodes:
cumin -b 5 '*' 'paoc'
## Disaster recovery
The LDAP server is mostly built by hand and should therefore be
restored from backups in case of a catastrophic failure. Care should
be taken to keep the SSH keys of the server intact.
The IP address (and name?) of the LDAP server should not be hard-coded
anywhere. When the server was last renumbered ([issue 33908](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33908)), the
only changes necessary were on the server itself, in `/etc`. So in
theory, a fresh new server could be deployed (from backups) in a new
location (and new address) without having to do much.
# Reference
## Installation
All `ud-ldap` components are deployed through Debian packages,
compiled from the git repositories. It is assumed that some manual
configuration was performed on the main LDAP server to get it
bootstrapped, but that procedure was lost in the mists of time.
Only backups keep us from total catastrophe in case of
lost. Therefore, this system probably cannot be reinstalled from
scratch.
## SLA
The LDAP server is designed to be fault-tolerant in the sense that
it's database is copied over other hosts. It should otherwise be
highly available as it's a key component in managing users
authentication and authorization, and machines.
## Design
The LDAP setup at Tor is based on the one from Debian.org. It has a
long, old and complex history, lost in the mists of time.
Configuration and database files like SSH keys, OpenPGP keyrings,
password, group databases, or email forward files are synchronised to
various hosts from the LDAP database. Most operations can be performed
on the [db.torproject.org](https://db.torproject.org/) site or by [email](https://db.torproject.org/doc-mail.html).
### Architecture overview
This is all implemented by a tool called `ud-ldap`, inherited from the
Debian project. The project is made of a collection of bash, Python
and Perl scripts which take care of synchronizing various
configuration files to hosts based on the LDAP configuration. Most of
this section aims at documenting how this program works.
`ud-ldap` is made of two Debian packages: `userdir-ldap`, which ships
the various server- and client-side scripts (and is therefore
installed everywhere), and `userdir-ldap-cgi` which ships the web
interface (and is therefore installed only on the LDAP server).
Configuration files are generated on the server by the `ud-generate`
command, which goes over the LDAP directory and crafts a tree of
configuration files, one directory per host defined in LDAP. Then each
host pulls those configuration files with `ud-replicate`. A common set
of files is exported everywhere, while the `exportOptions` field can
override that by disabling some exports or enabling special ones.
An email gateway processes OpenPGP-signed emails which can change a
user's fields, passwords or SSH keys, for example.
In general, `ud-ldap`:
* creates UNIX users and groups on (some or all) machines
* distributes password files for those users or other services
* distributes user SSH public keys
* distributes all SSH host public keys to all hosts
* configures and reload arbitrary services, but particularly handles
email, DNS, and git servers
* provides host metadata to Puppet
This diagram covers those inter-dependencies at the time of writing.
![LDAP architecture diagram](ldap/graph.svg)
### Configuration file distribution
An important part of `ud-ldap` is the `ud-generate` command, which
generates configuration files for each host. Then the `ud-replicate`
command runs on each node to `rsync` those files. Both commands are
ran from cron on regular intervals. `ud-replicate` is configured by
the `userdir-ldap` package, at every 5 minutes. `ud-generate` is also
configured to run every 5 minutes, starting on the third minute of
every hour, in `/etc/cron.d/local-ud-generate` (so at minute 3, 8, 13,
..., 53, 58).
More specifically, this is what happens:
1. on the LDAP server (currently `alberti`), `ud-generate` writes
various files (detailed below) in one directory per host
2. on all hosts, `ud-replicate` `rsync`'s that host's directory from
the LDAP server (as the `sshdist` user)
`ud-generate` will write files only if the LDAP database or keyring
changed since last time, or at most every 24 hours, based on the
timestamp (`last_update.trace`). The `--force` option can be used to
bypass those checks.
### Files managed by ud-generate
This is a (hopefully) exhaustive list of files generated by
`ud-generate` as part of userdir-ldap 0.3.97 ("UNRELEASED"). This
might have changed since this was documented, on 2020-10-07.
All files are written in the `/var/cache/userdir-ldap/hosts/`, with
one subdirectory per host.
| Path | Function | Fields used |
|------------------------------------|-----------------------------------------------------------------------------------|-------------------------------------------------------------------------------------------------------------------------------------|
| `all-accounts.json` | JSON list of users | `uid`, `uidNumber`, `userPassword`, `shadowExpire` |
| `authorized_keys` | `authorized_keys` file for `ssh_dist`, if `AUTHKEYS` in `exportOptions` | `ipHostNumber`, `sshRSAHostKey`, `purpose`, `sshdistAuthKeysHost` |
| `bsmtp` | ? | ? |
| `debian-private` | debian-private mailing list subscription | `privateSub`, `userPassword` (skips inactive) , `supplementaryGid` (skips guests) |
| `debianhosts` | list of all IP addresses, unused | `hostname`, `ipHostNumber` |
| `disabled-accounts` | list of disabled accounts | `uid`, `userPassword` (*includes* inactive) |
| `dns-sshfp` | per-host DNS entries (e.g. debian.org), if `DNS` in `exportOptions` | see below |
| `dns-zone` | user-managed DNS entries (e.g. debian.net), if `DNS` in `exportOptions` | `dnsZoneEntry` |
| `forward.alias` | .forward compatibility, unused? | `uid`, `emailForward` |
| `group.tdb` | `group` file template, with only the group that have access to that host | `uid`, `gidNumber`, `supplementaryGid` |
| `last_update.trace` | timestamps of last change to LDAP, keyring and last `ud-generate` run | N/A |
| `mail-callout` | ? | mailCallout |
| `mail-contentinspectionaction.cdb` | how to process this user's email (blackhole, markup, reject) | `mailContentInspectionAction` |
| `mail-contentinspectionaction.db` | | |
| `mail-disable` | disabled email messages | `uid`, `mailDisableMessage` |
| `mail-forward.cdb` | .forward "CDB" database, see [cdbmake(1)][] | `uid`, `emailForward` |
| `mail-forward.db` | .forward Oracle Berkeley DB "DBM" database | `uid`, `emailForward` |
| `mail-greylist` | greylist the account or not | mailGreylisting |
| `mail-rbl` | ? | mailRBL |
| `mail-rhsbl` | ? | mailRHSBL |
| `mail-whitelist` | ? | mailWhitelist |
| `markers` | xearth geolocation markers, unless `NOMARKERS` in `extraOptions` | `latitude`, `longitude` |
| `passwd.tbd` | `passwd` file template, if `loginShell` is set and user has access | `uid`, `uidNumber`, `gidNumber`, `gecos`, `loginShell` |
| `mail-passwords` | secondary password for mail authentication | `uid`, `mailPassword`, `userPassword` (skips inactive), `supplementaryGid` (skips guests) |
| `rtc-passwords` | secondary password for RTC calls | `uid`, `rtcPassword`, `userPassword` (skips inactive), `supplementaryGid` (skips guests) |
| `shadow.tdb` | `shadow` file template, same as `passwd.tdb`, if `NOPASSWD` not in `extraOptions` | `uid`, `uidNumber`, `userPassword`, `shadowExpire`, `shadowLastChange`, `shadowMin`, `shadowMax`, `shadowWarning`, `shadowInactive` |
| `ssh-gitolite` | `authorized_keys` file for `gitolite`, if `GITOLITE` in `exportOptions` | `uid`, `sshRSAAuthKey` |
| `ssh-keys-$HOST.tar.gz` | SSH *user* keys, as a tar archive | `uid`, `allowed_hosts` |
| `ssh_known_host` | SSH host keys | `hostname`, `sshRSAHostKey`, `ipHostNumber` |
| `sudo-passwd` | `shadow` file for `sudo` | `uid`, `sudoPassword` |
| `users.oath` | TOTP authentication | `uid`, `totpSeed`, `userPassword` (skips inactive) , `supplementaryGid` (skips guests) |
| `web-passwords` | secondary password database for web apps, if `WEB-PASSWORDS` in `extraOptions` | `uid`, `webPassword` |
### How files get distributed by ud-replicate
The `ud-replicate` program runs on all hosts every 5 minutes and logs
in as the `sshdist` user on the LDAP server. It rsyncs the files from
the `/var/cache/userdir-ldap/hosts/$HOST/` directory on the LDAP server to
the `/var/lib/misc/$HOST` directory.
For example, for a host named `example.torproject.org`, `ud-generate`
will write the files in
`/var/cache/userdir-ldap/hosts/example.torproject.org/` and
`ud-replicate` will synchronize that directory, on
`example.torproject.org`, in the
`/var/lib/misc/example.torproject.org/` directory. The
`/var/lib/misc/thishost` symlink will also point to that directory.
Then ud-replicate those special things with some of those
files. Otherwise consumers of those files are expected to use them
directly in `/var/lib/misc/thishost/`, as is.
#### `makedb` template files
Files labeled with `template` are inputs for the [makedb(1)][]
command. They are like their regular "non-template" counterparts,
except they have a prefix that corresponds to:
1. an incremental index, prefixed by zero (e.g. 01, 02, 03,
... 010...)
2. the `uid` field (the username), prefixed by a dot (e.g. `.anarcat`)
3. the `uidNumber` field (the UNIX UID), prefixed by an equal sign
(e.g. `=1092`)
Those are the fields for the `passwd` file. The `shadow` file has only
prefixes 1 and 2. This file format is used to create the databases in
`/var/lib/misc/` which are fed into the NSS database with the
[libnss-db](https://tracker.debian.org/pkg/libnss-db) package. The database files get generated by
[makedb(1)][] from the templates above. It is what allows the `passwd`
file in `/etc/passwd` to remain untouched while still allowing ud-ldap
to manage extra users.
[makedb(1)]: https://manpages.debian.org/makedb.1
#### self-configuration: sshdist `authorized_keys`
The `authorized_keys` file gets shipped if `AUTHKEYS` is set in
`extraOptions`. This is typically set on the LDAP server (currently
`alberti`), so that all servers can login to the server (as the
`sshdist` user) and synchronise their configuration with
`ud-replicate`.
This file gets dropped in `/var/lib/misc/authorized_keys` by
`ud-replicate`. A symlink in `/etc/ssh/userkeys/sshdist` ensures those
keys are active for the `sshdist` user.
#### other special files
More files are handled specially by `ud-replicate`:
* `forward-alias` gets modified (`@emailappend` appended to each
line) and replaces `/etc/postfix/debian`, which gets rehashed by
`postmap`. this is done only if `/etc/postfix` and `forward-alias`
exist
* the `bsmtp` config file is deployed in `/etc/exim4`, if both exist
* if `dns-sshfp` or `dns-zone` are changed, the DNS server zone files
get regenerated and server reloaded (`sudo -u dnsadm
/srv/dns.torproject.org/bin/update`, see "DNS zone file management"
below)
* `ssh_known_hosts` gets symlinked to `/etc/ssh`
* the `ssh-keys.tar.gz` tar archive gets decompressed in
`/var/lib/misc/userkeys`
* the `web-passwords` file is given to `root:www-data` and made
readable only by the group
* the `rtc-passwords` file is installed in `/var/local/` as:
* `rtc-passwords.freerad` if `/etc/freeradius` exists
* `rtc-passwords.return` if `/etc/reTurn` exists
* `rtc-passwords.prosody` if `/etc/prosody` exists
.. and the appropriate service (`freeradius`,
`resiprocate-turn-server`, `prosody`, respectively) get reloaded
### Authentication mechanisms
ud-ldap deals uses multiple mechanisms to authenticate users and
machines.
1. the web interface binds to the LDAP directory anonymously, or as
the logged in user, if any. an encrypted copy of the
username/password pair is stored on disk, encrypted, and passed
around in a URL token
2. the email gateway runs as the `sshdist` user and binds to the LDAP
directory using the `sshdist`-specific password. the `sshdist`
user has full admin rights to the LDAP database through the slapd
configuration. commands are authenticated using OpenPGP
signatures, checked against the keyring, maintained outside of
LDAP, manually, in the `account-keyring.git` repository, which
needs to be pushed to the LDAP server by hand.
3. `ud-generate` runs as the `sshdist` user and binds as that user
to LDAP as well
4. `ud-replicate` runs as root on all servers. it authenticates with
the central LDAP server over SSH *using the SSH server **host**
private key as a user key*, and logs in to the SSH server as the
`sshdist` user. the `authorized_keys` file for that user on the
LDAP server (`/etc/ssh/userkeys/sshdist`) determines which files
the client has access to using a predefined `rsync` command which
restricts to only `/var/cache/userdir-ldap/hosts/$HOST/`
5. Puppet binds to the LDAP server over LDAPS using the custom CA,
anonymously
6. LDAP admins also have access to the LDAP server directly, provided
they can get a shell (or a port forward) to access it
This is not related to ud-ldap authentication itself, but ud-ldap
obviously distributes authentication systems all over the place:
* PAM and NSS usernames and passwords
* SSH user authentication keys
* SSH server public keys
* `webPassword`, `rtcPassword`, `mailPassword`, and so on
* email forwards and email block list checks
* DNS zone files (which may include things like SSH server public
keys, for example)
### SSH access controls
A user gets granted access if it is part of a group that has been
granted access on the host with the `allowedGroups` field. An
additional group has access to *all* host, defined as
`allowedgroupspreload` (currently `adm`) in
`/etc/userdir-ldap/userdir-ldap.conf` on the LDAP server (currently
`alberti`).
Also note the `NOPASSWD` value for `exportOptions`: if set, it marks
the host as not allowing passwords so the `shadow` database is not
shipped which makes it impossible to login to the host with a
password. In practice this has no effect since password-based
authentication is disabled at the SSH server level, however.
### LDAP user fields
Those are the fields in the `user` LDAP object as of userdir-ldap
0.3.97 ("UNRELEASED"). This might have changed since this was
documented, on 2020-10-07. Some of those fields, but not all, can be
modified or deleted by the user through the email interface
(`ud-mailgate`).
| User field | Meaning |
| ---------- | ------- |
| `cn` | "common name" AKA "last name" |
| `emailForward` | address to forward email to |
| `gecos` | GECOS metadata field |
| `gidNumber` | Primary numeric group identifier, the UNIX GID |
| `homeDirectory` | UNIX `$HOME` location, unused |
| `ircNick` | IRC nickname, informative |
| `keyFingerprint` | OpenPGP fingerprint, grants access to email gateway |
| `labeledURI` | home page? |
| `loginShell` | UNIX login shell, grants user shell access, depending on gidNumber; breaks login if the corresponding package is not installed (ask TPA and see a related discussion in tpo/tpa/team#40854) |
| `mailCallout` | enables Sender Address Verification |
| `mailContentInspectionAction` | how to process user's email detected as spam (reject, blackhole, markup) |
| `mailDefaultOptions` | enables the "normal" set of SMTP checks, e.g. greylisting and RBLs |
| `mailGreylisting` | enables greylisting |
| `mailRBL` | set of RBLs to use |
| `mailRHSBL` | set of RHSBLs to use |
| `mailWhitelist` | sender envelopes to whitelist |
| `mailDisableMessage` | message to bounce messages with to disable an email account |
| `mailPassword` | [crypt(3)][]-hashed password used for email authentication |
| `rtcPassword` | previously used in XMPP authentication, unused |
| `samba*` | many samba fields, unused |
| `shadowExpire` | `1` if the account is expired |
| `shadowInactive` | ? |
| `shadowLastChange` | Last change date, in days since epoch |
| `shadowMax` | ? |
| `shadowMin` | ? |
| `shadowWarning` | ? |
| `sn` | "surname" AKA "first name" |
| `sshRSAAuthKey` | SSH public keys |
| `sudoPassword` | `sudo` passwords on different hosts |
| `supplementaryGid` | Extra groups GIDs the user is a member of |
| `uidNumber` | Numeric user identifier, the UNIX UID, not to be confused with the above |
| `uid` | User identifier, the user's *name* |
| `userPassword` | LDAP password field, stripped of the `{CRYPT}` prefix to be turned into a UNIX password if relevant |
[crypt(3)]: https://manpages.debian.org/crypt.3
[cdbmake(1)]: https://manpages.debian.org/cdbmake.1
#### sudoPassword field format
The `sudoPassword` field is special. It has 4 fields separated by
spaces:
1. a UUID
2. the status, which is either the string `unconfirmed` or the string
`confirmed:` followed by a SHA1 (!) HMAC of the string
`password-is-confirmed`, `sudo`, the UID, the UUID, the host list,
and the hashed password, joined by colons (`:`), primed with a
secret key stored in `/etc/userdir-ldap/key-hmac-$UID` where UID
is the numeric identifier of the calling user, generally `33`
(probably the web server?) or `sshdist`? The secret key can also
overridden by the `UD_HMAC_KEY` environment variable
3. the host list, either `*` (meaning all hosts) or a comma (`,`)
separated list of hosts this password applies to
4. the hashed password, which is restricted to 50 characters: if
longer, it is invalid (`*`)
That password field gets validated by email through `ud-mailgate`.
The field can, of course, have multiple values.
#### sshRSAAuthKey field format
The `sshRSAAuthKey` field can have multiple values. Each one should be
a valid [authorized_keys(5)][] file.
Its presence influences whether a user is allowed to login to a host
or not. That is, if it is missing, the user will not be added to the
`shadow` database.
The `GITOLITE` hosts treat the field specially: it looks for
`allowed_hosts` fields and will match only on the right host. If will
skip keys that have other options.
[authorized_keys(5)]: https://manpages.debian.org/authorized_keys.5
### LDAP host fields
Those are the fields in the `user` LDAP object as of userdir-ldap
0.3.97 ("UNRELEASED"). This might have changed since this was
documented, on 2020-10-07. Those fields are usually edited by hand by
an LDAP admin using `ldapvi`.
| Group field | Meaning |
|-----------------|--------------------------------------------------------------|
| `description` | free-form text field description |
| `memory` | main memory size, with `M` suffix (unused?) |
| `disk` | main disk size, with `G` suffixed (unused?) |
| [`purpose`][] | like description but purpose of the host |
| `architecture` | CPU architecture (e.g. `amd64`) |
| `access` | always "restricted"? |
| `physicalHost` | parent metal or hoster |
| `admin` | always "torproject-admin@torproject.org" |
| `distribution` | always "Debian" |
| `l` | location ("City, State, Country"), unused |
| `ipHostNumber` | IPv4 or IPv6 address, multiple values |
| `sshRSAHostKey` | SSH server public key, multiple values |
| [`rebootPolicy`][] | how to reboot this server: `manual`, `justdoit`, `rotation`) |
[`purpose`]: #purpose-field-values
[`rebootPolicy`]: #rebootpolicy-field-values
#### `rebootPolicy` field values
The `rebootPolicy` is documented in the [reboot
procedures](howto/reboots).
#### `purpose` field values
The `purpose` field is special in that it supports a crude markup
language which can be used to create links in the web interface, but
is also used to generate SSH `known_hosts` files. To quote the
ud-generate source code:
> In the purpose field, `[[host|some other text]]` (where some other
> text is optional) makes a hyperlink on the web \[interface\]. We now
> also add these hosts to the ssh `known_hosts` file. But so that we
> don't have to add everything we link, we can add an asterisk and say
> `[[*...` to ignore it. In order to be able to add stuff to ssh
> without http linking it we also support `[[-hostname]]` entries.
Otherwise the `description` and `purpose` fields are fairly similar
and often contain the same value.
Note that there can be multiple `purpose` values, in case we need
multiple names like that. For example, the prometheus/grafana server
has:
```
purpose: [[-prometheus1.torproject.org]]
purpose: [[prometheus.torproject.org]]
purpose: [[grafana.torproject.org]]
```
because:
* `prometheus1.torproject.org`: is an SSH alias but not a web one
* `prometheus.torproject.org`: because the host also runs Prometheus
as a web interface
* `grafana.torproject.org`: and that is the Grafana web interface
Note that those do *not* (unfortunately) add a CNAME in DNS. That
needs to be done by hand in `dns/domains.git`.
#### `exportOptions` field values
The `exportOptions` field warrants a more detailed explanation. Its
value determines which files are created by `ud-generate` for a given
host. It can either enable or inhibit the creation of certain files.
* `AUTHKEYS`: ship the `authorized_keys` file for `sshdist`,
typically on the LDAP server for `ud-replicate` to connect to it
* `BSMTP`: ship the `bsmtp` file
* `DNS`: ships DNS zone files (`dns-sshfp` and `dns-zone`)
* `GITOLITE`: ship the gitolite-specific SSH `authorized_keys`
file. can also be suffixed, e.g. `GITOLITE=OPTIONS` where `OPTIONS`
does magic stuff like skip some hosts (?) or change the SSH command
restriction
* `KEYRING`: ship the `sync_keyrings` GnuPG keyring file (`.gpg`)
defined in `userdir-ldap.conf`, generated from the
`admin/account-keyring.git` repository (technically: the
`ssh://db.torproject.org/srv/db.torproject.org/keyrings/keyring.git`
repository...)
* `NOMARKERS`: inhibits the creation of the `markers` file
* `NOPASSWD`: if present, the `passwd` database has `*` in the
password field, `x` otherwise. also inhibits the creation of the
`shadow` file. also marks a host as `UNTRUSTED` (below)
* `PRIVATE`: ship the `debian-private` mailing list registration file
* `RTC-PASSWORDS`: ship the `rtc-passwords` file
* `MAIL-PASSWORDS`: ship the `mail-passwords` file
* `TOTP`: ship the `users.oath` file
* `UNTRUSTED`: skip sudo passwords for this host unless explicitly
set
* `WEB-PASSWORDS`: ship the `web-passwords` file
Of those parameters, only `AUTHKEYS`, `DNS` and `GITOLITE` are used at
TPO, for, respectively, the LDAP server, DNS servers, and the git
server.
### Email gateway
The email gateway runs on the LDAP server. There are four aliases,
defined in `/etc/aliases`, which forward to the `sshdist` user with an
extension:
change: sshdist+changes
changes: sshdist+changes
chpasswd: sshdist+chpass
ping: sshdist+ping
Then three `.forward` files in the `~sshdist` home directory redirect
this to the `ud-mailgate` Python program while also appending a copy
of the email into `/srv/db.torproject.org/mail-logs/`, for example:
# cat ~sshdist/.forward+changes
"| /usr/bin/ud-mailgate change"
/srv/db.torproject.org/mail-logs/received.changes
This is how `ud-mailgate` processes incoming messages:
1. it parses the email from stdin using Python's `email.parser`
library
2. it tries to find an OpenPGP-signed message and passes it to the
`GPGCheckSig` function to verify the signature against the trusted
keyring
3. it does a check against replay attacks by checking:
* if the OpenPGP signature timestamp is reasonable (less than 3
days in the future, or 4 days in the past)
* if the signature has already been received in the last 7 days
The `ReplayCache` is a [dbm](https://docs.python.org/3/library/dbm.html) database stored in
`/var/cache/userdir-ldap/mail/replay`.
4. it then behaves differently whether it was called with `ping`,
`chpass` or `change` as its argument
5. in any case it tries to send a reply to the user by email,
encrypted in the case of `chpass`
The `ping` routine just responds to the user with their LDAP entry,
rendered according to the `ping-reply` template (in
`/etc/userdir-ldap/templates`).
The `chpass` routine behaves differently depending on a magic string
in the signed message, which can either be:
1. "Please change my Debian password"
2. "Please change my Tor password"
3. "Please change my Kerberos password"
4. "Please change my TOTP seed"
The first two do the same thing. The latter two are not in use at
TPO. The main `chpass` routine basically does this:
1. generate a 15-character random string
2. "hash" it with Python's [crypt](https://docs.python.org/3/library/crypt.html) with a MD5 (!) salt
4. set the hashed password in the user's LDAP object, `userPassword`
field
5. bump the `shadowLastChange` field in the user's LDAP object
6. render the `passwd-changed` email template which will include an
OpenPGP encrypted copy of the cleartext email
The `change` routine does one or many of the following, depending on
the lines in the signed message:
* on `show`: send a `key: value` list of parameters of the user's
LDAP object, OpenPGP-encrypted
* change the user's "position marker" (latitude/longitude) with a
format like `Lat: -10.0 Long: +10.0`
* add or replace a `dnsZoneEntry` if the line looks like `host IN
{A,AAAA,CNAME,MX,TXT}`
* replace LDAP user object fields if the line looks like `field:
value`. only some fields are supported
* add or replace `sshRSAAuthKey` lines when the line looks like an
SSH key (note that this routine sends its error email
separately). this gets massaged so that it matches the format
expected by `ud-generate` in LDAP and is validated by piping in
`ssh-keygen -l -f`. the `allowed_hosts` block is checked against
the existing list of servers and it enforces a minimum RSA key size
(2048 bits)
* delete an LDAP user field, when provided with a line that looks
like `del FIELD`
* add or replace `mailrbl`, `mailrhsbl` and `mailwhiltelist` fields,
except allow a space separator instead of the normal colon
separator for arbitrary fields (??)
* if the sudo password is changed, it checks if the HMAC provided
matches the expected one from the database and switched from
`unconfirmed` to `confirmed`
Note that the `change` routine only operates if the account is not
locked (if the `userPassword` does not contain the string `*LK*` or
starts with the `!` string).
### Web interface
The web interface is shipped as part of the userdir-ldap-cgi Debian
package, built from the [userdir-ldap-cgi repository][]. The web
interface is written in Perl, using the builtin CGI module and [WML][]
templates. It handles password and settings changes for users,
although some settings (like sudo passwords) require an extra
confirmation by OpenPGP-signed message through the email gateway. It
also lists machines known by LDAP.
[userdir-ldap-cgi repository]: https://gitlab.torproject.org/tpo/tpa/userdir-ldap-cgi/
The web interface also ships documentation in the form of HTML pages
rendered through [WML][] templates.
The web interface binds to the LDAP database as the logged in user (or
anonymously, for some listings and searches) and therefore doesn't
enjoy any special privilege in itself.
Each "dynamic" page is a standalone CGI script, although it uses some
common code from `Util.pm` to load settings, format some strings, deal
with authentication tokens and passwords.
The main page is the `search.cgi` interface, which allows users to
perform a search in the user database, based on a subset of LDAP
fields. This script uses the `searchform.wml` template.
The login form (`login.cgi`) binds with the LDAP database using the
provided user/password. A "hack" is present to "upgrade" the user's
passwords to MD5, presumably it was in cleartext
before. Authentication persistence is done through an authentication
token (`authtoken` in the URL), which consists of a MD5 "*encoded
username and a key to decrypt the password stored on disk, the
authtoken is protected from modification by an HMAC*". In practice, it
seems the user's password is stored on disk, encrypted with a Blowfish
cipher in CBC mode (from `Crypt::CBC`), with a 10 bytes (80 bits) key,
while the HMAC is based on SHA1 (from `Digest::HMAC_SHA1`). The tokens
are stored in `/var/cache/userdir-ldap/web-cookies/` with one file per
user, named after a salted MD5 hash of the username. Tokens expire
after 10 minutes by the web interface, but it doesn't seem like old
tokens get removed unless the user is active on the site.
Although the user/password pair is not stored directly in the user's
browser cookies or history, the authentication token effectively acts
as a valid user/password to make changes to the LDAP user database. It
could be abused to authenticate as an LDAP user and change their
password, for example.
The login form uses the `login.wml` template.
The `logout.cgi` interface, fortunately, allows users to clear this
on-disk data, invalidating possibly leaked tokens.
The `update.cgi` interface is what processes actual changes requested
by users. It will extract the actual LDAP user and password from the
on-disk encrypted token and bind with that username and password. It
does some processing of the form to massage it into a proper LDAP
update, running some password quality checks using a wrapper around
[cracklib](https://github.com/cracklib/cracklib) called `password-qualify-check` which, essentially,
looks at a word list, the GECOS fields and the old password. Partial
updates are possible: if (say) the `rtcPassword` fields don't match
but the `userPassword` fields do, the latter will be performed because
it is done first. It is here that unconfirmed `sudo` passwords are set
as well. It's the user's responsibility to send the challenge response
by signed OpenPGP email afterwards. This script uses the `update.wml`
template.
The `machines.cgi` script will list servers registered in the LDAP in
a table. It binds to the LDAP server anonymously and searches for all
hosts. It uses the `hostinfo.wml` template.
Finally the `fetchkey.cgi` script will load a public key from the
`keyrings` configuration setting based on the provided fingerprint and
dump it in plain text.
### Interactions with Puppet
The [Puppet server](howto/puppet) is closely coupled with LDAP, from which
it gathers information about servers.
It specifically uses those fields:
| LDAP field | Puppet use |
|----------------|-------------------------------------------------------------------------------------------------|
| `hostname` | matches with the Puppet node host name, used to load records |
| `ipHostNumber` | Ferm firewall, Bind, Bacula, PostgreSQL backups, static sync access control, backends discovery |
| `purpose` | motd |
| `physicalHost` | motd: shows parent in VM, VM children in host |
The `ipHostnumber` field is also used to lookup the host in the
`hoster.yaml` database in order to figure out which hosting provider
hosts the parent metal. This is, in turn, used in Hiera to change
certain parameters, like Debian mirrors.
Note that the above fields are explicitly imported in the
`allnodeinfo` data structure, along with `sshRSAHostKey` and
`mXRecord`, but those are not used. Furthermore, the `nodeinfo`
data structure imports all of the host's data, so there might be other
fields in use that I haven't found.
Puppet connects to the LDAP server directly over LDAPS (port 636) and
therefore requires the custom LDAP host CA, although it binds to the
server anonymously.
### DNS zone file management
One of the configuration files `ud-generate` generates are,
critically, the `dns-sshfp` and `dns-zone` files.
The `dns-sshfp` file holds the following records mapped to LDAP
`host` fields:
| DNS record | LDAP host field | Notes |
| ---------- | --------------- | ----- |
| `SSHFP` | `sshRSAHostKey` | extra entries possible with the `sshfphostname` field |
| `A`, `AAAA` | `ipHostNumber` | TTL overridable with the `dnsTTL` field |
| `HINFO` | `architecture` and `machine` | |
| `MX` | `mXRecord` | |
The `dns-zone` file contains *user*-specific DNS entries. If a `user`
object has a `dnsZoneEntry` field, that entry is written to the file
directly. A `TXT` record with the user's email address and their PGP
key fingerprint is also added for identification. That file is not in
use in TPO at the moment, but is (probably?) the mechanism behind the
user-editable `debian.net` zone.
Those files only get *distributed* to DNS servers (e.g. `nevii` and
`falax`), which are marked with the `DNS` flag in the `exportOptions`
field in LDAP.
Here is how zones are propagated from LDAP to the DNS server:
1. `ud-replicate` will pull the files with `rsync`, as explained in
the previous section
2. if the `dns-zone` or `dns-sshfp` files change, `ud-replicate` will
call `/srv/dns.torproject.org/bin/update` (from `dns_helpers.git`)
as the `dnsadm` user, which creates the final zonefile in
`/srv/dns.torproject.org/var/generated/torproject.org`
The `bin/update` script does the following:
1. pulls the `auto-dns.git` and `domains.git` git repositories
2. updates the DNSSEC keys (with `bin/update-keys`)
3. update the GeoIP distribution mechanism (with `bin/update-geo`)
4. builds the service includes from the `auto-dns` directory (with
`auto-dns/build-services`), which writes the
`/srv/dns.torproject.org/var/services-auto/all` file
5. for each domain in `domains.git`, calls `write_zonefile` (from
`dns_helpers.git`), which in turn:
1. increments the serial number in the `.serial` state file
2. generate a zone header with the new serial number
3. include the zone from `domains.git`
4. compile it with [named-compilezone(8)][], which is the part
that expands the various `$INCLUDE` directives
6. then calls `dns-update` (from `dns_helpers.git`) which rewrites
the `named.conf` snippet and reloads bind, if needed
[named-compilezone(8)]: https://manpages.debian.org/named-compilezone.8
The various `$INCLUDE` directives in the `torproject.org` zonefile are
currently:
* `/var/lib/misc/thishost/dns-sshfp` - generated on the LDAP server
by `ud-generate`, contains SSHFP records for each host
* `/srv/dns.torproject.org/puppet-extra/include-torproject.org`:
generated by Puppet modules which call the `dnsextras` module. This
is used, among other things, for TLSA records for HTTPS and SMTP
services
* `/srv/dns.torproject.org/var/services-auto/all`: generated by the
`build-services` script in the `auto-dns.git` directory
* `/srv/letsencrypt.torproject.org/var/hook/snippet`: generated by
the `bin/le-hook` in the `letsencrypt-domains.git` repository, to
authenticate against Let's Encrypt and generate [TLS](howto/tls)
certificates.
Note that this procedure fails when the git server is unavailable, see
[issue 33766](https://gitlab.torproject.org/tpo/tpa/team/-/issues/33766) for details.
### Source file analysis
Those are the various scripts shipped by userdir-ldap. This table
describes which programming language it's written in and a short
description of its purpose. The `ud?` column documents whether the
command was considered for implementation in the [ud](https://github.com/Debian/ud) rewrite, and
gives us a hint on whether it is important or not.
| tool | lang | ud? | description |
| ------------------------- | ------ | --- | --------------------------------------------------------------- |
| `ud-arbimport` | Python | | import arbitrary entries into LDAP |
| `ud-config` | Python | | prints config from `userdir-ldap.conf`, used by `ud-replicate` |
| `ud-echelon` | Python | x | "Watches for email activity from Debian Developers" |
| `ud-fingerserv` | Perl | x | [finger(1)][] server to expose some (public) user information |
| `ud-fingerserv2.c` | C | | same in C? |
| `ud-forwardlist` | Python | | convert `.forward` files into LDAP configuration |
| `ud-generate` | Python | x | critical code path, generates all configuration files |
| `ud-gpgimport` | Python | | seems unused? "Key Ring Synchronization utility" |
| `ud-gpgsigfetch` | Python | | refresh signatures from a keyring? unused? |
| `ud-groupadd` | Python | x | tries to create a group, possibly broken, not implemented by ud |
| `ud-guest-extend` | Python | | "Query/Extend a guest account" |
| `ud-guest-upgrade` | Python | | "Upgrade a guest account" |
| `ud-homecheck` | Python | | audits home directory permissions? |
| `ud-host` | Python | | interactively edits host entries |
| `ud-info` | Python | | same with user entries |
| `ud-krb-reset` | Perl | | kerberos password reset, unused? |
| `ud-ldapshow` | Python | | stats and audit on the LDAP database |
| `ud-lock` | Python | x | locks many accounts |
| `ud-mailgate` | Python | x | email operations |
| `ud-passchk` | Python | | audit a password file |
| `ud-replicate` | Bash | x | rsync file distribution from LDAP host |
| `ud-replicated` | Python | | rabbitmq-based trigger for ud-replicate, unused? |
| `ud-roleadd` | Python | x | like ud-groupadd, but for roles, possibly broken too |
| `ud-sshlist` | Python | | like ud-forwardlist, but for ssh keys |
| `ud-sync-accounts-to-afs` | Python | | sync to AFS, unused |
| `ud-useradd` | Python | x | create a user in LDAP, possibly broken? |
| `ud-userimport` | Python | | imports passwd and group files |
| `ud-xearth` | Python | | generates xearth DB from LDAP entries |
| `ud-zoneupdate` | Shell | x | increments serial on a zonefile and reload bind |
Note how the `ud-guest-upgrade` command works. It generates an LDAP
snippet like:
delete: allowedHost
-
delete: shadowExpire
-
replace: supplementaryGid
supplementaryGid: $GIDs
-
replace: privateSub
privateSub: $UID@debian.org
where the `guest` gid is replaced by the "default" `defaultgroup`
set in the `userdir-ldap.conf` file.
[finger(1)]: https://manpages.debian.org/finger.1
Those are other files in the source distribution which are not
directly visible to users but are used as libraries by other files.
| libraries | lang | description |
| ----------------------- | ------ | -------------------------------------------- |
| `UDLdap.py` | Python | mainly an Account representation |
| `userdir_exceptions.py` | Python | exceptions |
| `userdir_gpg.py` | Python | yet another GnuPG Python wrapper |
| `userdir_ldap.py` | Python | various functions to talk with LDAP and more |
Those are the configuration files shipped with the package:
| configuration files | lang | description |
| ---------------------------- | ------ | ----------------------------------------------------------------------- |
| `userdir-ldap.conf` | Python | LDAP host, admin user, email, logging, keyrings, web, DNS, MX, and more |
| `userdir_ldap.pth` | ??? | no idea! |
| `userdir-ldap.schema` | LDAP | TPO/Debian-specific LDAP schema additions |
| `userdir-ldap-slapd.conf.in` | slapd | slapd configuration, includes LDAP access control |
## Issues
There is no issue tracker specifically for this project, [file][] or
[search][] for issues in the [team issue tracker][search], with the
~LDAP label.
[file]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/new
[search]: https://gitlab.torproject.org/tpo/tpa/team/-/issues?label_name%5B%5D=LDAP
## Maintainer, users, and upstream
Our [userdir-ldap repository][] is a fork of the [DSA userdir-ldap
repository][]. The codebase is therefore shared with the Debian
project, which uses it more heavily than TPO. According to [GitLab's
analysis](https://salsa.debian.org/dsa-team/mirror/userdir-ldap/-/graphs/master), weasel has contributed the most to the repository (since
2007), followed closely by Joey Schulze, which wrote most of the code
before that, between 1999 and 2007.
[DSA userdir-ldap repository]: https://salsa.debian.org/dsa-team/mirror/userdir-ldap
[userdir-ldap repository]: https://gitlab.torproject.org/tpo/tpa/userdir-ldap/
The service is mostly in maintenance mode, both at DSA and in TPO,
with small, incremental changes being made to the codebase over all
those years. Attempts have been made to rewrite it with a Django
frontend ([ud](https://github.com/Debian/ud), 2013-2014 no change since 2017) or Pylons
([userdir-ldap-pylons](https://salsa.debian.org/dsa-team/mirror/userdir-ldap-pylons), 2011, abandoned), all have been abandoned.
Our fork is primarily maintained by anarcat and weasel. It is used by
*everyone* at Tor.
Our fork tries to follow upstream as closely as possible, but the
Debian project is hardcoded in a lot of places so we (currently) are
forced to keep patches on top of upstream.
### Branching policy
In the [userdir-ldap][userdir-ldap repository] and [userdir-ldap-cgi repository][], we have
tried to follow the [icebreaker branching strategy used at one of
Google's kernel teams](https://lwn.net/Articles/871195/). Briefly, the idea is to have patches
rebased on top of the latest upstream release, with each feature
branch based on top of the tag. Those branches get merged in our
"master" branch which contains our latest source code. When a new
upstream release is done, a new feature branch is created by merging
the previous feature branch and the new release.
See page 24 and page 25 of the [talk slides](https://static.sched.com/hosted_files/osselc21/f1/OSS%202021%20-%20Icebreaker.pdf) for a view of what
that graph looks like. This is what it looks like in `userdir-ldap`:
```
$ git log --decorate --oneline --graph --all
* 97c5660 (master) Merge branch 'tpo-scrub-0.3.104-pre'
|\
| * 698da3a (tpo-scrub-0.3.104-pre-dd7f9a3) update changelog after rebase
| * b05f7d0 Set emailappend to torproject.org
| * 407775c Use https:// in welcome email
| * fecc816 Re-apply tpo changes to Debian's repo
| * dd7f9a3 (dsa/master) ud-mailgate: fix SPF verification logic to work correctly with "~all"
| * f991671 Actually ship ud-guest-extend
```
In this case, there is only one feature branch left, and it's now
identical to `master`.
This is what it looks like in `userdir-ldap-cgi`:
```
* 25cf477 (master) Merge branch 'tpo-scrub-0.3.43-pre-5091066'
|\
| * 0982aa0 (tpo-scrub-0.3.43-pre-5091066) remove debian-specific stylesheets, use TPO
| * 5eb5da8 remove email features not enabled on torproject.org
| * 54c03de remove direct access note, disabled in our install
| * fec1282 Removed lines which mention finger (TPO has no finger services)
| * 18f3aeb drop many fields from update form
| * d1dd377 Replace "debian" with "torproject" as much as possible
| * 7dcc1a1 (clean-series-0.3.43-pre-5091066) add keywords in changes mail commands help
| * aecb3c8 use an absolute path in SSH key upload
| * ca110ab remove another needless use of cat
| * 685f36b use relative link for web form, drop SSL
| * b7bd99d don't document SSH key changes in the password lost page (#33134)
| * 05a10e5 explicitly state that we do not support pgp/mime (#33134)
| * f98bba6 clarify that show requires a signature as well (#33134)
| * e41d911 suggest using --sign for the SSH key as well (#33134)
| * 50933fd improve sudo passwords update confirmation string
| * 2907fc2 add spacing in doc-mail
| * 5091066 (dsa/master) Update now broken links to the naming scheme page to use archive.org
| * c08a063 doc-direct: stop referring to access changes from 2003
```
In this particular case the `tpo-scrub` branch is based on top of the
`clean-series` patch because there would be too many conflicts
otherwise (and we are really, really hoping the patches can be
merged). But typically those would both be branched off `dsa/master`.
This pattern is designed so that it's easier to send patches
upstream. Unfortunately, upstream releases are somewhat irregular so
this somewhat breaks down because we don't have a solid branch point
to base our feature branches off. This is why the branches are named
like `tpo-scrub-0.3.104-pre-dd7f9a3`: the `pre-dd7f9a3` is to indicate
that we are not branched off a real release.
TODO: consider git's newer `--update-refs` to see if it may help
maintain those branches, see [this post](https://andrewlock.net/working-with-stacked-branches-in-git-is-easier-with-update-refs/)
Update: as of 2025-04-17, we have mostly abandoned trying to merge
patches upstream after yet again other releases produced upstream that
have not merged our patches. See the [2025 update](#2025-update) below.
### usedir-ldap-cgi fork status
In the [last sync](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40182), `usedir-ldap-cgi` was brought from 27 patches
down to 16, 10 of which were sent upstream. Our diff there is now:
22 files changed, 11661 insertions(+), 553 deletions(-)
The large number of inserted lines is because we included the
[styleguide](https://styleguide.torproject.org) `bootstrap.css` which is 11561 lines on its own, so
really, this is the diff stat if we ignore that stylesheet:
21 files changed, 100 insertions(+), 553 deletions(-)
If the patches get merged upstream, our current delta is:
21 files changed, 23 insertions(+), 527 deletions(-)
Update: none of our recent patches were merged upstream. We still have
the following branches:
* `auth-status-code-0.3.43`: send proper codes on authentication
failures, to enable `fail2ban` parsing
* `mailpassword-update-0.3.43`: enables mail password edits on the
web interface
* `clean-series-0.3.43`: various cleanups
* `tpo-scrub-0.3.43`: `s/debian.org/torproject.org/`, TPO-specific
* `feature-pretty-css-0.3.43`: CSS cleanups and UI tweaks, TPO-specific
Apart from getting patches merged upstream, the only way forward here
is either to make the "Debian" strings "variables" in the WML
templates or completely remove the documentation from userdir-ldap-cgi
(and move it to the project's respective wikis).
For now, we have changed the navigation to point to our wiki as much
as possible. The next step is to remove our patches to the upstream
documentation and make sure that documentation is not reachable to
avoid confusion.
### userdir-ldap fork status
Our diff in `userdir-ldap` used to be much smaller (in 2021):
6 files changed, 46 insertions(+), 19 deletions(-)
We had 4 patches there, and a handful were merged upstream. The
remaining patches could probably live as configuration files in
Puppet, reducing the diff to nil.
#### 2023 update
Update, 2023-05-10: some patches were merged, some weren't, and we had
to roll new ones. We have the following diff now:
```
debian/changelog | 22 ++++++++++++++++++++++
debian/compat | 2 +-
debian/control | 5 ++---
debian/rules | 3 +--
debian/ud-replicate.cron.d | 2 +-
templates/passwd-changed | 2 +-
templates/welcome-message | 41 ++++++++++++++++++++++++++++-------------
test/test_pass.py | 10 ++++++++++
ud-mailgate | 5 +++--
ud-replicate | 11 +++++++++--
userdir-ldap.conf | 2 +-
userdir_ldap/UDLdap.py | 5 +++++
userdir_ldap/generate.py | 22 +++++++++++++++++++++-
userdir_ldap/ldap.py | 2 +-
14 files changed, 106 insertions(+), 28 deletions(-)
```
We now have *five* branches left:
* `tpo-scrub-0.3.104`:
* `43c67a3` fix URL in passwd-changed template to torproject.org
* `f9f9a67` Set emailappend to torproject.org
* `c77a70b` Use https:// in welcome email
* `6966895` Re-apply tpo changes to Debian's repo
* `mailpassword-generate-0.3.104`:
* `6b09f95` distribute mail-passwords in a location dovecot can read
* `666c050` expand mail-password file fields
* `5032f73` add simple getter to Account
* `hashpass-test-0.3.104`, `7ceb72b` add tests for ldap.HashPass
* `bookworm-build-0.3.104`:
* `25d89bd` fix warning about chown(1) call in bookworm
* `9c49a4a` fix Depends to support python3-only installs
* `1ece069` bump dh compat to 7
* `90ef120` make this build without python2
* `ssh-sk-0.3.104`, `a722f6f` Add support for security key generated ssh public keys (sk- prefix)
The rebase was done with the following steps.
First we laid down a tag because upstream didn't:
```
git tag 0.3.104 81d0512e87952d75a249b277e122932382b86ff8
```
Then we created new branches for each old branch and rebased it on
that release:
```
git checkout -b genpass-fix-0.3.104 origin/genpass-fix-0.3.104-pre-dd7f9a3
git rebase 0.3.104
git branch -m hashpass-test-0.3.104
git checkout -b procmail-0.3.104 procmail-0.3.104-pre-dd7f9a3
git rebase 0.3.104
git branch -d procmail-0.3.104
git checkout -b mailpassword-generate-0.3.104 origin/mailpassword-generate-0.3.104-pre-dd7f9a3
git rebase 0.3.104
git checkout -b tpo-scrub-0.3.104 origin/tpo-scrub-0.3.104-pre-dd7f9a3
git rebase 0.3.104
git checkout master
git merge hashpass-test-0.3.104
git merge mailpassword-generate-0.3.104
git merge tpo-scrub-0.3.104
git checkout -b bookworm-build-0.3.104 0.3.104
git merge bookworm-build-0.3.104
```
Verifications of the resulting diffs were made with:
```
git diff master dsa
git diff master origin/master
```
Then the package was built and tested on `forum-test-01`, `chives`,
`perdulce` and `alberti`:
```
dpkg-buildpackage
```
And finally uploaded to db.tpo and git:
```
git push origin -u hashpass-test-0.3.104
git push origin -u mailpassword-generate-0.3.104
git push origin -u bookworm-build-0.3.104 0.3.104
git push origin -u tpo-scrub-0.3.104
git push
```
Eventually, we merged with upstream's master branch to be able to use
micah's patch (in
https://gitlab.torproject.org/tpo/tpa/team/-/issues/41166), so we
added an extra branch in there.
#### 2024 update
As of 2024-06-03, the situation has not improved:
```
anarcat@angela:userdir-ldap$ git diff dsa/master --stat
.gitlab-ci.yml | 18 ------------------
debian/changelog | 22 ++++++++++++++++++++++
debian/rules | 2 +-
debian/ud-replicate.cron.d | 2 +-
misc/ud-update-sudopasswords | 4 ++--
templates/passwd-changed | 2 +-
templates/welcome-message | 41 ++++++++++++++++++++++++++++-------------
test/test_pass.py | 10 ++++++++++
ud-mailgate | 14 ++++++++------
ud-replicate | 4 ++--
userdir-ldap.conf | 2 +-
userdir_ldap/generate.py | 49 ++++++++++++++++++++++++++++++++++++++-----------
12 files changed, 114 insertions(+), 56 deletions(-)
```
We seem incapable of getting our changes merged upstream at this
point. Numerous patches were sent to DSA only to be either ignored,
rewritten, or replaced without attribution. It has become such a
problem that we have effectively given up on merging the two
code bases.
We should acknowledge that *some* patches were actually merged, but
the patches that weren't were so demotivating that it seems easier to
just track this as a non-collaborating upstream, with our code as a
friendly fork, than pretending there's real collaboration happening.
Our patch set is currently:
* `tpo-scrub-0.3.104` (unchanged, possibly unmergeable):
* `43c67a3` fix URL in passwd-changed template to torproject.org
* `f9f9a67` Set emailappend to torproject.org
* `c77a70b` Use https:// in welcome email
* `6966895` Re-apply tpo changes to Debian's repo
* `mailpassword-generate-0.3.104` (patch rewritten upstream, unclear
if still needed)
* `hashpass-test-0.3.104` (unchanged)
* 7ceb72b (add tests for ldap.HashPass, 2021-10-27 15:29:30 -0400)
* `fix-crash-without-exim-0.3.104` (new)
- 51716ed (ud-replicate: fix crash when exim is not installed, 2023-05-11 13:53:33 -0400)
* `paramiko-workaround-0.3.104-dff949b` (new, not sent upstream
considering `ssh-openssh-87` was rejected)
- 6233f8e (workaround SSH host key lookup bug in paramiko, 2023-11-21 14:49:46 -0500)
* `sshfp-openssh-87` (new, rejected)
- 651f280 (disable SSHFP record for initramfs keys, 2023-05-10 14:38:56 -0400)
* `py3_allowed_hosts_unicode-0.3.104)` (new, rewritten upstream, conflicting)
- 88bb60d (LDAP now returns bytes, fix another comparison in ud-mailgate, 2023-10-12 10:23:53 -0400)
* `thunderbird-sequoia-pgp-0.3.105` (new)
* 4cb6d49 (extract PGP/MIME multipart mime message content
correctly, 2024-06-03)
* 417f78b (fix Sequoia signature parsing, 2024-06-03)
* ddc8553 (fix Thunderbird PGP/MIME support, 2024-06-03)
Existing patches were not resent or rebased, but *were* sent upstream
unless otherwise noted.
The following patches were actually merged:
* `bookworm-build-0.3.104`:
* d0740a9 (fix implicit int to str cast that broke in bookworm (bullseye?) upgrade, 2023-09-13)
* `25d89bd` fix warning about chown(1) call in bookworm
* `9c49a4a` fix Depends to support python3-only installs
* `1ece069` bump dh compat to 7
* `90ef120` make this build without python2
* `install-restore-crash-0.3.104`:
* 4ab5d83 (fix crash: LDAP returns a string, cast it to an integer, 2023-09-14 10:28:41 -0400)
* `procmail-0.3.104-pre-dd7f9a3`:
* 661875e (drop procmail from userdir-ldap dependencies, 2022-02-28 21:15:41 -0500)
This patch are still in development:
* `ssh-sk-0.3.104`
* a722f6f Add support for security key generated ssh public keys
(sk- prefix).
It should also be noted that some changes are sitting naked on
`master`, without feature branches and have not been submitted
upstream. Those are the known cases but there might be others:
* 91e5b2f (add backtrace to ud-mailgate errors, 2024-06-05)
* 65555da (fix crash in sudo password changes@, 2024-06-05)
* 4315593 (fix changes@ support, 2024-06-05)
* 76a22f0 (note the thunderbird patch merge, 2024-06-04)
* e90f16e (add missing sshpubkeys dependency, 2024-06-04)
* d2cb1d4 (fix passhash test since SHA256 switch, 2024-06-04)
* b566604 (make_hmac expects bytes, convert more callers, 2023-09-28)
* f24a9b5 (remove broken coverage reports, 2023-09-28)
#### 2025 update
We had to do an emergency merge to cover for trixie, which upstream
added support for recently. We were disappointed to see the
`thunderbird-sequoia-pgp-0.3.105` and `fix-crash-without-exim-0.3.104`
ignored upstream, and another patch rejected.
At this point, we're treating our fork as a downstream and are not
trying to contribute back upstream anymore. Concretely, this meant the
`thunderbird-sequoia-pgp-0.3.105` patch broke and had to be dropped
from the tree. Other changes were also committed directly to master
and not sent upstream, in particular:
* 9edccfa (fix error on fresh install, 2025-04-17)
* 8c4a9f5 (deal with ud-replicate clients newer than central server, 2025-04-17)
Next step is probably planning for ud-ldap retirement and replacement,
see [tpo/tpa/team#41839](https://gitlab.torproject.org/tpo/tpa/team/-/issues/41839) and [TPA-RFC-86](https://gitlab.torproject.org/tpo/tpa/team/-/wikis/policy/tpa-rfc-86-identity-access-management).
## Monitoring and testing
Prometheus checks the `/var/lib/misc/thishost/last_update.trace` timestamp
and warns if a host is more than an hour out of date.
The web and mail servers are checked as per normal policy.
## Logs and metrics
The LDAP directory holds a list of usernames, email addresses, real
names, and possibly even physical locations. This information gets
destroyed when a user is completely removed but can be kept
indefinitely for locked out users.
`ud-ldap` keeps a full copy of all emails sent to
`changes@db.torproject.org`, `ping@db.torproject.org` and
`chpasswd@db.torproject.org` in `/srv/db.torproject.org/mail-logs/`. This
includes personally identifiable information (PII) like `Received-by`
headers (which may include user's IP addresses), user's email
addresses, SSH public keys, hashed sudo passwords, and junk mail. The
mail server should otherwise follow normal mail server logging
policies.
The web interface keeps authentication tokens in
`/var/cache/userdir-ldap/web-cookies`, which store encrypted username
and password information. Those get removed when a user logs out or
after 10 minutes of inactivity, when the user returns. It's unclear
what happens when a user forgets to logout and fails to return to the
site. Web server logs should otherwise follow the normal TPO policy,
see [the static mirror network](howto/static-component#logs-and-metrics) for more information on that.
The OpenLDAP server itself (`slapd`) keeps no logs.
There are no performance metrics recorded for this service.
## Backups
There's no special backup procedures for the LDAP server, it is
assumed that the on-disk `slapd` database can be backed up reliably by
Bacula.
## Other documentation
* our (TPA) [userdir-ldap repository][]
* our (TPA) [userdir-ldap-cgi repository][]
* the [DSA wiki](https://dsa.debian.org/) has some ud-ldap documentation, see in
particular:
* [how to get sudo passwords](https://dsa.debian.org/user/sudo/)
* [how to add a guest account](https://dsa.debian.org/howto/add-guest/)
* [how to add an account](https://dsa.debian.org/howto/add-account/)
* [how to lock an account](https://dsa.debian.org/howto/lock-account/)
* upstream (DSA) [userdir-ldap source code](https://salsa.debian.org/dsa-team/mirror/userdir-ldap)
* upstream (DSA) [userdir-ldap-cgi source code](https://salsa.debian.org/dsa-team/mirror/userdir-ldap-cgi)
* [ud](https://github.com/Debian/ud) - a partial ud-ldap rewrite in Django from 2013-2014, no
change since 2017, the [announcement for the rewrite](https://wiki.debian.org/Teams/DSA/UserdirLdapRewrite)
* [userdir-ldap-pylons](https://salsa.debian.org/dsa-team/mirror/userdir-ldap-pylons) - a partial ud-ldap rewrite in Pylons from
2011, abandoned
* [LDAP.com](https://ldap.com/) has [extensive documentation](https://ldap.com/learn-about-ldap/), for example on
[LDAP filters](https://ldap.com/ldap-filters/)
# Discussion
## Overview
This section aims at documenting issues with the software and possible
alternatives.
`ud-ldap` is decades old (the `ud-generate` manpage mentions 1999, but
it could be older) and is hard to maintain, debug and extend.
It might have serious security issues. It is a liability, in the long
term, in particular for those reasons:
* **old cryptographic primitives**: SHA-1 is used to hash `sudo`
passwords, MD5 is used to hash user passwords, those hashes are
communicated over OpenPGP_encrypted email but stored in LDAP in
clear-text. There is a "hack" present in the web interface to
enforce MD5 passwords on logins, and the mail interface also has
MD5 hard-coded for password resets. Blowfish and HMAC-SHA-1 are also
used to store and authenticate (respectively) LDAP passwords in the
web interface. MD5 is used to hash usernames.
* **rolls its own crypto**: `ud-ldap` ships its own wrapper around GnuPG,
implementing the (somewhat arcane) command-line dialect. it has not
been determined if that implementation is either accurate or safe.
* **email interface hard to use**: it has trouble with standard
OpenPGP/MIME messages and is hard to use for users
* **old web interface**: it's made of old Perl CGI scripts that uses
a custom template format built on top of [WML][] with custom
pattern replacement, without any other framework than Perl's
builtin `CGI` module. it uses in-URL tokens which could be
vulnerable to XSS attacks.
[WML]: https://en.wikipedia.org/wiki/Website_Meta_Language
* **large technical debt**
* ud-ldap is written in (old) Python 2, Perl and shell. it will at
least need to be ported to Python 3 in the short term.
* code reuse is minimal across the project.
* ud-ldap has no test suite, linting or CI of any form.
* opening some files (e.g. `ud-generate`) yield so many style
warnings that my editor (Emacs with Elpy) disables checks.
* it is believed to be impossible or at least impractical to setup
a new ud-ldap setup from scratch.
* **authentication is overly complex**: as detailed in the
[authentication section](#authentication-mechanisms), with 6 different authentication
methods with the LDAP server.
* **replicates configuration management**: ud-ldap does configuration
management and file distribution, as root
(`ud-generate`/`ud-replicate`), something which should be reserved
to Puppet. this might have been justified when ud-ldap was written,
in 1999, since configuration management wasn't very popular back
then ([Puppet](https://en.wikipedia.org/wiki/Puppet_(software)) was created in 2005, only [cfengine](https://en.wikipedia.org/wiki/CFEngine) existed
back then, which was created in 1993)
* **difficult to customize**: Tor-specific customizations are made as
patches to the git repository and require a package rebuild. they
are therefore difficult to merge back upstream and require us to
run our own fork.
Our version of ud-ldap has therefore diverged from upstream. The
changes are not extensive, but they are still present and require a
merge every time we want to upgrade the package. At the time of
writing, it is:
anarcat@curie:userdir-ldap(master)$ git diff --stat f1e89a3
debian/changelog | 18 ++++++++++++++++++
debian/rules | 2 +-
debian/ud-replicate.cron.d | 2 +-
templates/welcome-message | 41 ++++++++++++++++++++++++++++-------------
ud-generate | 3 ---
ud-mailgate | 2 ++
ud-replicate | 2 +-
userdir-ldap-slapd.conf.in | 4 ++--
userdir-ldap.conf | 2 +-
userdir-ldap.schema | 9 ++++++++-
10 files changed, 62 insertions(+), 23 deletions(-)
It seems that upstream doesn't necessarily run released code, and we
certainly don't: the above merge point had 47 commits on top of the
previous release (0.3.96). The current release, as of October 2020, is
0.3.97, and upstream already has 14 commits on top of it.
The web interface is in a similar conundrum, except worse:
22 files changed, 192 insertions(+), 648 deletions(-)
At least the changes there are only on the HTML templates. The merge
task is tracked in [issue 40062](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40062).
## Goals
The goal of the current discussion would be to find a way to fix the
problems outlined above, either by rewriting or improving ud-ldap,
replacing parts of it, or replacing ud-ldap completely with something
else, possibly removing LDAP as a database altogether.
### Must have
* framework in use must be supported for the foreseeable future
(e.g. not Python 2)
* unit tests or at least upstream support must be active
* system must be simpler to understand and diagnose
* single source of truth: overlap with Puppet must be
resolved. either Puppet uses LDAP as a source of truth (e.g. for
hosts and users) or LDAP goes away. compromises are possible:
Puppet could be the source of truth for hosts, and LDAP for users.
### Nice to have
* use one language across the board (e.g. Python 3 everywhere)
* reuse existing project's code, for example an existing LDAP
dashboard or authentication system
* ditch LDAP. it's hard to understand and uncommon enough to cause
significant confusion for users.
### Non-Goals
* we should avoid writing our own control panel, if possible
## Approvals required
The proposed solution should be adopted unanimously by TPA. A survey
might be necessary to confirm our users would be happy with the change
as well.
## Proposed Solution
TL;DR: three phase migration away from LDAP
1. stopgap: merge with upstream, port to Python 3 if necessary
2. move hosts to Puppet, replace ud-ldap with another user dashboard
3. move users to Puppet (sysadmins) or Kubernetes / GitLab CI /
GitLab Pages (developers), remove LDAP and replace with SSO dashboard
The long version...
### Short term: merge with upstream, port to Python 3 if necessary
In the **short term**, the situation with Python 2 needs to be
resolved. Either the Python code needs to be ported to Python 3, or it
needs to be replaced by something else. That is "urgent" in the sense
that Python 2 is already end of life and will likely not be supported
by the next Debian release, around summer 2024. Some work in that
direction has been done upstream, but it's currently unclear whether
ud-ldap is or will be ported to Python 3 in the short term.
The **diff with upstream** also makes it hard to collaborate. We
should make it possible to use directly the upstream package with a
local configuration, without having to ship and maintain our own fork.
Update: there has been progress on both of those fronts. Upstream
ported to Python 3 (partially?), but scripts (e.g. `ud-generate`)
still have the `python2` header. Preliminary tests seem to show that
`ud-generate` might be capable of running under `python3` directly as
well (ie. it doesn't error).
The diff with upstream has been reduced, see [upstream section for
details](#maintainer-users-and-upstream).
### Mid term: move hosts to Puppet, possibly replace ud-ldap with simpler dashboard
In the **mid-term**, we should remove the duplication of duty
between Puppet and LDAP, at least in terms of actual file
distribution, which should be delegated to Puppet. In practical terms,
this implies replacing `ud-generate` and `ud-replicate` with the
Puppet server and agents. It could still talk with LDAP for the host
directory, but at that point it might be better to simply **move all
host metadata into Hiera**.
It would still be nice to retain a dashboard of sorts to show the
different hosts and their configurations. Right now this is
accomplished with the [machines.cgi](https://db.torproject.org/machines.cgi) web interface, but this could
probably be favorably replaced by some static site generator. Gandi
implemented [hieraviz](https://github.com/Gandi/hieraviz) for this (now deprecated) and still maintain
a command-line tool called [hieracles](https://github.com/Gandi/hieracles) that somewhat overlaps with
[cumin](howto/cumin) and [hieraexplain](https://github.com/binford2k/hiera_explain) as well. Finally, a Puppet Dashboard
could replace this, see [issue tpo/tpa/team#31969](https://gitlab.torproject.org/tpo/tpa/team/-/issues/31969) for a discussion
on that, which includes the suggestion of moving the host inventory
display into Grafana, which has already started.
For users, the situation is less clear: we need some sort of dashboard
for users to **manage their email forward** and, if that project ever sees
the light of day, their email (submission, IMAP?) password. It is also
needed to **manage shell access** and SSH keys. So in the mid-term, the
**LDAP user directory would remain**.
At this point, however, it might not be necessary to use ud-ldap at
all: another dashboard could be use to manage the LDAP database. The
`ud-mailgate` interface could be retired and the web interface
replaced with something simpler, like [ldap-user-manager][ldap-user-manager].
So hopefully, in the mid term, it should be possible to completely
replace ud-ldap with Puppet for hosts and sysadmins, and an already
existing LDAP dashboard for user interaction.
### Long term: replace LDAP completely, with Puppet, GitLab and Kubernetes, possibly SSO dashboard
In the **long term**, the situation is muddier: at this stage, our
dependence on ud-ldap is either small (just users) or non-existent (we
use a different dashboard). But we still have LDAP, and that might be
a database we could get rid of completely.
We could simply stop offering shell access to non-admin users. User
access on servers would be managed completely by Puppet: only `sudo`
passwords need to be set for sysadmin anyways and those could live
inside Hiera.
Users currently requiring shell access would be encouraged to migrate
their service to a container image and workflow. This would be backed
by GitLab (for source code), GitLab CI/CD (for deployment) and
Kubernetes (for the container backend). Shell access would be limited
to sysadmins, which would take on orphan services which would be
harder to migrate inside containers.
Because the current shell access provided is very limited, it is
believe migration to containers would actually be not only feasible
but also beneficial for users, as they would possibly get more
privileges than they currently do.
Storage could be provided by Ceph and PostgreSQL clusters.
Those are the current services requiring shell access (as per
`allowedGroups` in the LDAP host directory), and their possible
replacements:
| Service | Replacement |
|---------------------------------------------|----------------------------------------------|
| Applications (e.g. bridgedb, onionoo, etc) | GitLab CI, Kubernetes or Containers |
| fpcentral | [retirement][] |
| Debian package archive | GitLab CI, GitLab pages |
| Email | email-specific dashboard |
| Git(olite) maintenance | GitLab |
| Git(web) maintenance | GitLab |
| Mailing lists | Debian packages + TPA |
| RT | Debian packages + TPA |
| Schleuder maintenance | Debian packages + TPA |
| Shell server (e.g. IRC) | ZNC bouncer in a container |
| Static sites (e.g. mirror network, ~people) | GitLab Pages, GitLab CI, Nginx cache network |
Those services were successfully replaced:
| Service | Replacement |
|---------|-------------|
| Jenkins | GitLab CI |
| Trac | GitLab |
Note that this implies the TPA team takes over certain services
(e.g. Mailman, RT and Schleuder, in the above list). It might mean
expanding the sysadmin team to grant access to service admins.
It also implies switching the email service to another, hopefully
simpler, dashboard. Alternatively, this could be migrated back into
Puppet as well: we already manage a lot of email forwards by hand in
there and we already get support requests for people to change their
email forward because they do not understand the ud-ldap interface
well enough to do it themselves (e.g. [this ticket](https://gitlab.torproject.org/tpo/tpa/team/-/issues/40059)). We could also
completely delegate email hosting to a third-party provider, as was
discussed in [the submission project](service/email).
Those are the applications that would need to be containerized for
this approach to be completed:
* BridgeDB
* Check/tordnsel
* Collector
* Consensus health
* CiviCRM
* Doctor
* Exonerator
* Gettor
* Metrics
* OnionOO
* Survey
* Translation
* ZNC
[retirement]: https://gitlab.torproject.org/tpo/tpa/team/-/issues/40009
This is obviously a quite large undertaking and would need to be
performed progressively. Thankfully, it can be done in parallel
without having to convert everything in one go.
Alternatively, a single-sign-on dashboard like [FreeIPA][] or
[Keycloak][] could be considered, to unify service authentication and
remove the plethora of user/password pairs we use everywhere. This is
definitely not being served by the current authentication system
(LDAP) which basically offers us a single password for all services
(unless we change the schema to add a password for each new service,
which is hardly practical).
## Cost
This would be part of the running TPA budget.
## Alternatives considered
The LDAP landscape in the free world is somewhat of a wasteland,
thanks to the "embrace and extend" attitude Microsoft has taken to the
standard (replacing LDAP and Kerberos with their proprietary Active
Directory standard).
### Replacement web interfaces
* [eGroupWare][]: has an LDAP backend, probably not relevant
* [LDAP account manager][]: self-service interface non-free
* [ldap-user-manager][]: "PHP web-based interface for LDAP user
account management and self-service password change", seems
interesting
* [GOsa][]: "administration frontend for user administration"
* [phpLDAPadmin][]: like [phpMyAdmin][] but for LDAP, for "power users",
long history of critical security issues
* [web2ldap][]: web interface, python, still maintained, not exactly intuitive
* [Fusion Directory](https://www.fusiondirectory.org/)
[phpMyAdmin]: https://www.phpmyadmin.net/
[ldap-user-manager]: https://github.com/wheelybird/ldap-user-manager
It might be simpler to rewrite `userdir-ldap-cgi` with [Django][], say
using the [django-auth-ldap][] authentication plugin.
[web2ldap]: https://web2ldap.de/
[eGroupWare]: https://www.egroupware.org/en/
[phpLDAPadmin]: http://phpldapadmin.sourceforge.net/
[GOsa]: https://github.com/gosa-project/gosa-core
[LDAP account manager]: https://www.ldap-account-manager.org/lamcms/
### Command-line tools
* [cpu][]: "Change Password Utility", with an LDAP backend, no
release since 2004
* [ldapvi][]: currently in use by sysadmins
* [ldap-utils][]: is part of OpenLDAP, has utilities like `ldapadd`
and `ldapmodify` that work on LDIF snippets, like `ldapvi`
* [shelldap][]: similar to `ldapvi`, but a shell!
* [splatd][]: syncs `.forward`, SSH keys, home directories, abandoned
for 10+ years?
[cpu]: http://cpu.sourceforge.net/
[splatd]: https://github.com/threerings/splatd
[shelldap]: https://hg.sr.ht/~mahlon/shelldap
[ldapvi]: http://www.lichteblau.com/ldapvi/
[ldap-utils]: https://packages.debian.org/unstable/ldap-utils
### Rewrites
* [netauth](https://www.netauth.org/) "can replace LDAP and Kerberos to provide
authentication services to a fleet of Linux machines. The Void
Linux project uses NetAuth to provide authentication securely over
the internet"
### Single-sign on
"Single-sign on" (SSO) is "an authentication scheme that allows a user
to log in with a single ID to any of several related, yet independent,
software systems." -- [Wikipedia](https://en.wikipedia.org/wiki/Single_sign-on)
In our case, it's something that could allow all our applications that
use a single source of truth for usernames and passwords. We could
also have a single place to manage the 2FA configurations, so that
users wouldn't have to enroll their 2FA setup in each application
individually.
Here's a list of the possible applications that could do this that
we're aware of:
| Application | MFA | webauthn | OIDC | SAML | SCIM | LDAP | Radius | Notes |
|------------------|-----|----------|------|------|------|------|--------|------------------------------------------------------------------------------|
| [Authelia][] | 2FA | ✓ | ✓ | ✗ | ✗ | ✓ | ✗ | rate-limiting, password reset, HA, Go/React |
| [Authentik][] | 2FA | ✓ | ✓ | ✗ | ✗ | ✓ | ✓ | proxy, metrics, Python/TypScript, sponsored by DigitalOcean |
| [Casdoor][] | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | ✓ | CAS, sponsored by Stytch, widely used |
| [Dex][] | ✗ | ✗ | ✓ | ✓ | ✗ | ✓ | ✗ | |
| [FreeIPA][] | ✗ | ✗ | ✗ | ✗ | ✗ | ✓ | ✗ | DNS, web/CLI UI, C?, built on top of 389 DS (Fedora LDAP server) |
| [A/I id][] | 2FA | ✓ | ✗ | ✗ | ✗ | ✓ | ✗ | SASL, PAM, Proxy, SQLite, rate-limiting |
| [Kanidm][] | 2FA | ✗ | ✓ | ✗ | ✗ | ✓ | ✓ | SSH, PAM + offline support, web/CLI UI, Rust |
| [Keycloak][] | 2FA | [✗][w3c] | ✓ | 2 | ✗ | ✓ | ✗ | Kerberos, SQL, web UI, HA/clustering, Java, sponsored by RedHat |
| [LemonLDAP-ng][] | 2FA | ✓ | ✓ | ✓ | ✗ | ✓ | ✗ | Kerberos, SQL, Perl, [packaged in Debian][] |
| [obligator][] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | password less, anonymous OIDC |
| [ory.sh][] | 2FA | ✓ | ✓ | ✗ | ✗ | ✗ | ✗ | multi-tenant, account verification, password resets, HA, Golang, complicated |
| [portier][] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | mainly proxy, password less/resets, replacement for Mozilla Personas |
| [vouch-proxy][] | ✗ | ✗ | ✓ | ✗ | ✗ | ✗ | ✗ | proxy |
| [zitadel][] | ✓ | ✓ | ✓ | 2 | ✗ | ✓ | ✗ | multi-tenant, passkeys, |
See also [mod_auth_openidc](https://github.com/OpenIDC/mod_auth_openidc) for an Apache module supporting OIDC.
A solution could be to deploy Keycloak or some SSO server on *top* of
the current LDAP server to provide other applications with a single
authentication layer. Then the underlying backend could be changed to
swap ud-ldap out if we need to, replacing bits of it as we go.
[Authelia]: https://www.authelia.com/
[Authentik]: https://goauthentik.io/
[Kanidm]: https://github.com/kanidm/kanidm
[LemonLDAP-ng]: https://lemonldap-ng.org/
[packaged in Debian]: https://tracker.debian.org/pkg/lemonldap-ng
[w3c]: https://github.com/keycloak/keycloak-community/blob/main/design/web-authn-two-factor.md
[A/I id]: https://git.autistici.org/id
[ory.sh]: https://www.ory.sh/
[Dex]: https://dexidp.io/
[Casdoor]: https://casdoor.org/
[portier]: https://portier.github.io/
[zitadel]: https://zitadel.com/
[obligator]: https://github.com/lastlogin-io/obligator
[vouch-proxy]: https://github.com/vouch/vouch-proxy
#### Keycloak
Was briefly considered at Debian.org which ended up using GitLab as an
identity provider (!). Concerns raised:
* [this post](https://lists.debian.org/debian-project/2020/04/msg00006.html) mentions "jboss" and:
- no self service for group or even OIDC clients
- no U2F (okay, GitLab also still needs to make the step to webauthn)
See also [this discussion](https://lists.debian.org/debian-project/2020/04/msg00000.html) and [this one](https://lists.debian.org/debian-devel/2017/08/msg00465.html). Another [HN discussion](https://news.ycombinator.com/item?id=36384636).
#### LemonLDAP
https://lemonldap-ng.org/
* has a GPG plugin
### Others
* [LDAP synchronization connector][]: "Open source connector to
synchronize identities between an LDAP directory and any data
source, including any database with a JDBC connector, another LDAP
server, flat files, REST API..."
* [LDAPjs][]: pure Javascript LDAP client
* [GQLDAP][]: GTK client, abandoned
* [LDAP admin][]: Desktop interface, written in Lazarus/Pascal (!)
* [lldap][]: rust rewrite, incomplete LDAP implementation, has a
control panel
* [ldap-git-backup][]: pull `slapcat` backups in a git repository,
useful for auditing purposes, [expiration might be an issue](https://github.com/elmar/ldap-git-backup/issues/12)
[LDAP admin]: https://github.com/ibv/LDAP-Admin
[Django]: https://www.djangoproject.com/
[django-auth-ldap]: https://pypi.org/project/django-auth-ldap/
[GQLDAP]: https://sourceforge.net/projects/gqclient/
[LDAPjs]: http://ldapjs.org/
[FreeIPA]: https://www.freeipa.org/
[Keycloak]: https://www.keycloak.org/
[LDAP synchronization connector]: https://lsc-project.org/doku.php
[ldap-git-backup]: https://github.com/elmar/ldap-git-backup/issues/12
[lldap]: https://github.com/lldap/lldap
### SCIM
LDAP is a "open, vendor-neutral, industry standard application
protocol for accessing and maintaining distributed directory
information services over an Internet Protocol (IP) network"
([Wikipedia](https://en.wikipedia.org/wiki/Lightweight_Directory_Access_Protocol)). That's quite a mouthful but concretely, many systems
have used LDAP as a single source of truth for authentication, relying
on it as an external user database (to simplify).
But that's only one way to do centralized authentication, and some
folks are reconsidering that approach altogether. A recent player in
there is the [SCIM standard](https://en.wikipedia.org/wiki/System_for_Cross-domain_Identity_Management): "System for Cross-domain Identity
Management (SCIM) is a standard for automating the exchange of user
identity information between identity domains, or IT systems"
([Wikipedia](https://en.wikipedia.org/wiki/System_for_Cross-domain_Identity_Management)). Again quoting Wikipedia:
> One example might be that as a company onboards new employees and
> separates from existing employees, they are added and removed from
> the company's electronic employee directory. SCIM could be used to
> automatically add/delete (or, provision/de-provision) accounts for
> those users in external systems such as Google Workspace, Office
> 365, or Salesforce.com. Then, a new user account would exist in the
> external systems for each new employee, and the user accounts for
> former employees might no longer exist in those systems.
In other words, instead of treating the user database as an external
database, SCIM synchronizes that database to all systems which still
retain their own specific user database. This is great because it
removes the authentication system as a single point of failure.
SCIM is standardized as [RFC7643](https://www.rfc-editor.org/rfc/rfc7643.html) and is built on top of REST with
data formatted as JSON or XML.