We found that the order of keys in the known_hosts generated by
ud-ldap is non-deterministic and will sometimes put the (initramfs)
key first. This normally wouldn't be a problem: SSH considers all
keys when doing lookups and will accept any valid one specified in the
file.

But paramiko only takes the *first* key, which means that if the
"(initramfs)" key ends up first in the file, paramiko will fail to
connect most of the time.

Now, this fixes the common case by putting the initramfs key second,
but of course this will still fail if paramiko tries to connect while
the host is in its initramfs.

We seem to be playing whack-a-mole here, maybe a better fix would be
to fix the LDAP connection to properly decode bytes for all calls, but
for now let's just fix this one issue, hopefully.

This was tested by manually reproducing part of the code in
production, with:

    import ldap
    lc = ldap.initialize('ldap://db.torproject.org')
    lc.simple_bind_s("", "")
    HostBaseDn = "ou=hosts,dc=torproject,dc=org"
    Res = lc.search_s(HostBaseDn, ldap.SCOPE_SUBTREE, '(objectClass=debianServer)', ['hostname'])
    ValidHostNames = []
    for _, value in Res:
        for s in value.get("hostname", []):
            ValidHostNames.append(s.decode('utf-8'))
    'perdulce.torproject.org' in ValidHostNames

It would be pretty neat to have a mockup for the LDAP server that
would allow us to unit-test this, but for now we're just scratching
this painful itch, again.

Closes: team#41357

This crashes the pipelines with:

    jobs:test buster:artifacts:reports config contains unknown keys: cobertura

There were some conflicts on this merge, mainly around disagreements
with upstreams around patches I sent, for example:

https://salsa.debian.org/dsa-team/mirror/userdir-ldap/-/commit/40d26915d400e1af3721ab106032efe933503053

and

https://salsa.debian.org/dsa-team/mirror/userdir-ldap/-/commit/40aa479fe570e251646fcd8a3f7788a61b816ded

For the former, we revert back to their version, but for the latter,
we kept ours.

This is likely going to mess up our branches, but I am losing faith in
our capacity to meaningfully merge things upstream a this point, so
I'm just doing some damage control.

commit 25b3ad4f fixed this for ud-replicated,
but now we're hitting this in generate after upgrading to bullseye.

This fixes that crash which occurs on subsequent ud-generate runs:

Traceback (most recent call last):
  File "/usr/bin/ud-generate", line 51, in <module>
    ud_generate()
  File "/usr/lib/python3/dist-packages/userdir_ldap/generate.py", line 1723, in ud_generate
    need_update = (ldap_last_mod > cache_last_ldap_mod) or (unix_last_mod > cache_last_unix_mod) or (time_started - last_run > MAX_UD_AGE)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'str' and 'int'

I have done a crazy buster -> bookworm upgrade of torproject.org's
LDAP server, and it did as badly as you would expect.

One thing that happened is both the main and `cn=log` databases were
flushed. The main database was restored, but for some reason the
second database couldn't be restore, or to be more accurate was
restored as empty.

This led the `sorted_mods` list there to be completely empty, which
raised the following exception:

root@alberti:/var/backups# sudo -u sshdist ud-generate
Traceback (most recent call last):
  File "/usr/bin/ud-generate", line 51, in <module>
    ud_generate()
  File "/usr/lib/python3/dist-packages/userdir_ldap/generate.py", line 1715, in ud_generate
    ldap_last_mod = getLastLDAPChangeTime(lc)
                    ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/lib/python3/dist-packages/userdir_ldap/generate.py", line 1615, in getLastLDAPChangeTime
    last = sorted_mods[-1][1]['reqEnd'][0].split(b'.')[0].decode('ascii')
           ~~~~~~~~~~~^^^^
IndexError: list index out of range

That bit of defensive programming should work around such atrocities
in the future.

I'm not sure why we're using a string for one mod time and integers
for all the others, but it's just not working anymore in Debian
bookworm (and probably bullseye / Python 3+).

This should fix the following backtrace:

root@alberti:~# sudo -u sshdist ud-generate
Traceback (most recent call last):
  File "/usr/bin/ud-generate", line 51, in <module>
    ud_generate()
  File "/usr/lib/python3/dist-packages/userdir_ldap/generate.py", line 1722, in ud_generate
    need_update = (ldap_last_mod > cache_last_ldap_mod) or (unix_last_mod > cache_last_unix_mod) or (time_started - last_run > MAX_UD_AGE)
                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '>' not supported between instances of 'int' and 'str'

'dict_keys' object has no attribute 'sort'

The dbm module doesn't seem to want to open older database files.

This is intended to reduce the delta between the userdir_ldap instance
of the tor project and us. For now DSA won't setup any imap server.

Signed-off-by: Pierre-Elliott Bécue <peb@debian.org>

This will allow, eg, dovecot, to feed on it.

Signed-off-by: Pierre-Elliott Bécue <peb@debian.org>

As in dict.get(key, default_value)

Signed-off-by: Pierre-Elliott Bécue <peb@debian.org>

The patch was triggering issues with unrelated mailgate functionality,
see team#41196.

This reverts commit a722f6f4.

Without this, ud-replicate aborts halfway through with:

root@submit-01:/var/lib/misc# ud-replicate
receiving file list ... done
submit-01.torproject.org/last_update.trace

sent 44 bytes  received 936 bytes  1,960.00 bytes/sec
total size is 245,745  speedup is 250.76
chown: invalid group: ‘root:Debian-exim’

... unless exim4 is installed, which is not the case in the Tor
Project (we use Postfix).

There are similar guards elsewhere in the code, it seems it's just an
oversight in this case.

This issue only applies to hosts that do not have Exim installed *and*
are SMTP submission servers.

debian.org also has hosts that don't have Exim installed (lists and
salsa both use Postifx) and ud-replicate works fine on them (because
the code in question is never attempted on them).

Without this, ud-replicate aborts halfway through with:

root@submit-01:/var/lib/misc# ud-replicate
receiving file list ... done
submit-01.torproject.org/last_update.trace

sent 44 bytes  received 936 bytes  1,960.00 bytes/sec
total size is 245,745  speedup is 250.76
chown: invalid group: ‘root:Debian-exim’

... unless exim4 is installed, which is not the case in the Tor
Project (we use Postfix).

There are similar guards elsewhere in the code, it seems it's just an
oversight in this case.

micah authored Dec 26, 2022 and anarcat committed May 10, 2023

In 2020, OpenSSH 8.2 was released which supports FIDO (Fast Identity Online) U2F
security keys. The public key type is used by hardware tokens, such as (but not
limited to) Yubikeys, to generate a 'security key' (sk) public key of either
`ecdsa-sk` or `ed25519-sk` key pairs.

The `ed25519-sk` key type is only supported by new Yubikeys with firmware 5.2.3
or higher, which supports FIDO2. Keys with firmware below that version are only
compatible with `ecdsa-sk` key types.

This change builds on caa57689 which added
support for ecdsa-sha2-nistp256 keys, by also allowing for the `sk-` prefix for
public keys.

Ssh public keys generated on a security token will look like one of the
following:

sk-ecdsa-sha2-nistp256@openssh.com <public key material> <optional comment>

or:
sk-ssh-ed25519@openssh.com  <public key material> <optional comment>

Note: the DoSSH() in ud-mailgate.py checks to make sure that a provided public
key does not have newlines, but as implemented was only checking for three
different key types (`ssh-rsa`, `ssh-ed25519`, and `ecdsa-sha2-nistp256`)
userdir-ldap supports more types than those checked. Rather than just adding the
two `sk-` key types to this check, and ignoring the other missing key types, I
changed this check to mirror the regexp used to check for key types (with the
regexp addition to confirm the keytype occurs on its own line, to indicate
newlines were erroneously provided).

This is to mitigate an issue that came up with clients running OpenSSH
8.7 or later. Before that release, it was acceptable to have multiple
conflicting keys in DNS, just like it's okay to have multiple keys in
known_hosts files. But starting from this commit:

https://cvsweb.openbsd.org/cgi-bin/cvsweb/src/usr.bin/ssh/dns.c.diff?r1=1.40&r2=1.41

and this PR:

https://github.com/openssh/openssh-portable/pull/259

... the check is hardened. This seems to have been introduced thanks
to this bug report:

https://bugzilla.mindrot.org/show_bug.cgi?id=3322

The rationale was to warn about SHA1 records, but it actually warns
about any duplicate record, as far as I can tell.

An example of this problem is:

    anarcat@angela:~$ ssh dal-rescue-02.torproject.org
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    @    WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED!     @
    @@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
    IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
    Someone could be eavesdropping on you right now (man-in-the-middle attack)!
    It is also possible that a host key has just been changed.
    The fingerprint for the ED25519 key sent by the remote host is
    SHA256:l0wDT4pboFIfsZKrsreak6lGVQkPomc8GLdqYA25gf4.
    Please contact your system administrator.
    Update the SSHFP RR in DNS with the new host key to get rid of this message.
    Linux dal-rescue-02 5.10.0-21-amd64 #1 SMP Debian 5.10.162-1 (2023-01-21) x86_64

    [...]
    dal-rescue-02$

We mitigate this by simply omitting duplicate records in the first
place. We omit the initramfs entries which are the ones causing
trouble in our case. Those are still available through the shared
known_hosts file that doesn't trigger that problem.

This was giving us:

chown: warning: '.' should be ':': ‘root.shadow’