Skip to content
Snippets Groups Projects

cleanup LDAP vs manual UNIX groups in GitLab

    • Closed Issue created by anarcat @anarcat

    we have a bunch of users and groups in LDAP for gitlab-02:

    allowedGroups: dip
allowedGroups: git
allowedGroups: gitlab-consul
allowedGroups: gitlab-redis
allowedGroups: gitlab-www
allowedGroups: registry

    of those, i think only git is used, but only by name. indeed, in LDAP the git UID is 1504 while on gitlab-02 it is:

    root@gitlab-02:~# id git
uid=995(git) gid=2149(git) groups=2149(git)

    ouch.

    similarly, we have the gitlab-psql user in operation on gitlab-02, yet it's not in LDAP:

    root@gitlab-02:~# id gitlab-psql
uid=993(gitlab-psql) gid=2145(gitlab-psql) groups=2145(gitlab-psql)

    those are the processes actually running right now with anything ressembling a git username:

    root@gitlab-02:~# ls -dal /proc/*/ | grep git | sort | sed 's/Jun.*//' | sort -u
dr-xr-xr-x  9 git          git          0 
dr-xr-xr-x  9 gitlab-psql  git          0 
dr-xr-xr-x  9 gitlab-psql  gitlab-psql  0 
dr-xr-xr-x  9 gitlab-redis git          0 
dr-xr-xr-x  9 gitlab-redis gitlab-redis 0

    numerically, here are all the UIDs in use on gitlab-02 right now:

    root@gitlab-02:~# ls -alnd /proc/*/ | grep -v '0    0 0' | sed 's/Jun.*//' | sort -u
dr-xr-xr-x  9   1    1 0 
dr-xr-xr-x  9  33   33 0 
dr-xr-xr-x  9 104  105 0 
dr-xr-xr-x  9 107  113 0 
dr-xr-xr-x  9 108  114 0 
dr-xr-xr-x  9 109  115 0 
dr-xr-xr-x  9 110  117 0 
dr-xr-xr-x  9 112  120 0 
dr-xr-xr-x  9 113    0 0 
dr-xr-xr-x  9 114  122 0 
dr-xr-xr-x  9 119  127 0 
dr-xr-xr-x  9 120  128 0 
dr-xr-xr-x  9 993 2145 0 
dr-xr-xr-x  9 993 2149 0 
dr-xr-xr-x  9 994 2144 0 
dr-xr-xr-x  9 994 2149 0 
dr-xr-xr-x  9 995 2149 0 
dr-xr-xr-x  9 997  997 0 
dr-xr-xr-x  9 998  998 0

    so i don't think any of the LDAP users or groups are in use here, but I could be mistaken. In any case, this should be cleaned up. at the very least the dip user should be removed altogether.

    so here are the groups to check, and in there, each user that's a member should be checked as well:

    • allowedGroups: dip
    • allowedGroups: git
    • allowedGroups: gitlab-consul
    • allowedGroups: gitlab-redis
    • allowedGroups: gitlab-www
    • allowedGroups: registry

    for each of those:

    1. check that the group is not in use on gitlab-02
    2. check that all users part of that group are not in use
    3. check that no file is owned by the user or group
    4. delete the user and group in LDAP

    also do this:

    • check for unowned files on the server: find / /var/opt /srv/gitlab-backup/ /srv/gitlab-shared/ -xdev -nouser -o -nogroup
    • reboot the server to make sure everything works
    ✓ 8 of 8 checklist items completed · Edited
    To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information

    Child items ...

    • Activity

    • anarcat added Backlog label

      added Backlog label

    • anarcat assigned to @anarcat

      assigned to @anarcat

    • anarcat mentioned in commit wiki-replica@46a6e127

      mentioned in commit wiki-replica@46a6e127

    • anarcat added Next label and removed Backlog label

      added Next label and removed Backlog label

    • anarcat added Doing label and removed Next label

      added Doing label and removed Next label

    • anarcat
      anarcat @anarcat ·
      Author Owner

      of those, i think only git is used, but only by name. indeed, in LDAP the git UID is 1504 while on gitlab-02 it is:

      root@gitlab-02:~# id git
uid=995(git) gid=2149(git) groups=2149(git)

      ouch.

      similarly, we have the gitlab-psql user in operation on gitlab-02, yet it's not in LDAP:

      root@gitlab-02:~# id gitlab-psql
uid=993(gitlab-psql) gid=2145(gitlab-psql) groups=2145(gitlab-psql)

      a clarification on that. gitlab-psql is in LDAP, but it's UID:GID 2145:2145. the former doesn't seem to be in use, but the latter UID is in use, but it's also in /etc/group. so, in other words, the LDAP user/group can be safely removed, as long as it doesn't have files owned.

      it will be worth checking for unowned files on this server as well.

    • anarcat
      anarcat @anarcat ·
      Author Owner

      i found a bunch more accounts in LDAP that are probably not realized anywhere: any account with dip- as a prefix seems to not actually be in use, so i have just dropped them from LDAP altogether. here are the LDAPvi blobs:

      414 gid=dip-registry,ou=users,dc=torproject,dc=org
gid: dip-registry
gidNumber: 2130
objectClass: top
objectClass: debianGroup

415 gid=dip-signup,ou=users,dc=torproject,dc=org
gid: dip-signup
gidNumber: 2131
objectClass: top
objectClass: debianGroup

416 gid=dip-webhook,ou=users,dc=torproject,dc=org
gid: dip-webhook
gidNumber: 2132
objectClass: top
objectClass: debianGroup

417 uid=dip-registry,ou=users,dc=torproject,dc=org
uid: dip-registry
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
gidNumber: 2130
gecos: dip registry role,,,,
loginShell: /bin/false
cn: dip registry role
shadowLastChange: 17959
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
uidNumber: 2130

418 uid=dip-signup,ou=users,dc=torproject,dc=org
uid: dip-signup
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
gidNumber: 2131
gecos: dip signup role,,,,
loginShell: /bin/false
cn: dip signup role
shadowLastChange: 17959
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
uidNumber: 2131

419 uid=dip-webhook,ou=users,dc=torproject,dc=org
uid: dip-webhook
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
gidNumber: 2132
gecos: dip webhook role,,,,
loginShell: /bin/false
cn: dip webhook role
shadowLastChange: 17959
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
uidNumber: 2132

      those are, basically:

      gid=dip-registry gidNumber: 2130
gid=dip-signup gidNumber: 2131
gid=dip-webhook gidNumber: 2132
uid=dip-registry uidNumber: 2130
uid=dip-signup uidNumber: 2131
uid=dip-webhook uidNumber: 2132
    • anarcat
      anarcat @anarcat ·
      Author Owner

      i'm currently running this command in a screen to figure out if there are irrelevant users or groups in use:

      find / /var/opt /srv/gitlab-backup/ /srv/gitlab-shared/ -xdev -group dip -o -user gitlab-consul -o -group gitlab-consul  -o -user 2145 -o -user 1504 -o -user 2144 -o -user 2143 -o -user registry -o -group registry

      the last set of numeric users are:

      • 2143: gitlab-www user in LDAP, actually user 996 in /etc/passwd on gitlab-02
      • 2144: gitlab-redis in LDAP, user 994 in /etc/passwd
      • 2145: gitlab-psql, vs user 993

      the following groups have the right GID in /etc/group and do not need to be checked on the filesystem:

      • gitlab-www (2143)
      • gitlab-redis (2144)
      • gitlab-psql (2145)

      there is no uid=dip, strangely.

      that basically covers the entirety of the list of groups that's assigned to this host, mentioned in the issue summary:

          allowedGroups: dip
    allowedGroups: git
    allowedGroups: gitlab-consul
    allowedGroups: gitlab-redis
    allowedGroups: gitlab-www
    allowedGroups: registry

      the git user and group actually exist in /etc/passwd and /etc/group, but with 2149:2149. this doesn't correspond to the git user in LDAP, which is 1504:1504, so that's actually a problem. but the solution here is probably just to remove the allowedGroups line.

      the dip user is slightly more problematic: even if it doesn't have any owned files, it actually has members (ahf, dgoulet, hiro), so this would cut off access to the server to those people.

      @ahf, @dgoulet, it looks like you have shell access to the gitlab-02 server (like real shell, not git shell). this only grants you sudo access to the dip user, and i don't believe that can do anything. so i'm just going to remove that access now, i hope this won't break anything for you.

    • anarcat
      anarcat @anarcat ·
      Author Owner

      given that the git group had an incorrect UID in LDAP (or the reverse: the on-disk UID is incorrect on gitlab-02), i have removed it from the allowedGroups block from the host entry in LDAP.

      this should have no effect at all.

    • anarcat marked the checklist item allowedGroups: git as completed

      marked the checklist item allowedGroups: git as completed

    • anarcat changed the description

      changed the description

    • anarcat
      anarcat @anarcat ·
      Author Owner

      i'm currently running this command in a screen to figure out if there are irrelevant users or groups in use:

      find / /var/opt /srv/gitlab-backup/ /srv/gitlab-shared/ -xdev -group dip -o -user gitlab-consul -o -group gitlab-consul  -o -user 2145 -o -user 1504 -o -user 2144 -o -user 2143 -o -user registry -o -group registry -ls

      the last set of numeric users are:

      • 2143: gitlab-www user in LDAP, actually user 996 in /etc/passwd on gitlab-02
      • 2144: gitlab-redis in LDAP, user 994 in /etc/passwd
      • 2145: gitlab-psql, vs user 993

      this returned without any output, which means that the users and groups can be removed from LDAP.

    • anarcat
      anarcat @anarcat ·
      Author Owner

      i removed the following blobs:

      413 gid=dip,ou=users,dc=torproject,dc=org
gid: dip
gidNumber: 2129
objectClass: top
objectClass: debianGroup

455 gid=gitlab-www,ou=users,dc=torproject,dc=org
objectClass: top
objectClass: debianGroup
gidNumber: 2143
gid: gitlab-www

456 uid=gitlab-www,ou=users,dc=torproject,dc=org
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
shadowLastChange: 17085
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
loginShell: /bin/dash
gidNumber: 2143
uidNumber: 2143
uid: gitlab-www
cn: gitlab-www role account
gecos: gitlab-www role account,,,,

457 gid=gitlab-redis,ou=users,dc=torproject,dc=org
gid: gitlab-redis
objectClass: top
objectClass: debianGroup
gidNumber: 2144

458 uid=gitlab-redis,ou=users,dc=torproject,dc=org
uid: gitlab-redis
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
gecos: gitlab-redis role account,,,,
cn: gitlab-redis role account
shadowLastChange: 17085
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
loginShell: /bin/dash
gidNumber: 2144
uidNumber: 2144

459 gid=gitlab-psql,ou=users,dc=torproject,dc=org
gid: gitlab-psql
objectClass: top
objectClass: debianGroup
gidNumber: 2145

460 uid=gitlab-psql,ou=users,dc=torproject,dc=org
uid: gitlab-psql
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
gecos: gitlab-psql role account,,,,
cn: gitlab-psql role account
shadowLastChange: 17085
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
loginShell: /bin/dash
gidNumber: 2145
uidNumber: 2145

461 gid=registry,ou=users,dc=torproject,dc=org
gid: registry
objectClass: top
objectClass: debianGroup
gidNumber: 2146

462 uid=registry,ou=users,dc=torproject,dc=org
uid: registry
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
gecos: gitlab registry role account,,,,
cn: gitlab registry role account
shadowLastChange: 17085
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
loginShell: /bin/dash
gidNumber: 2146
uidNumber: 2146

463 gid=gitlab-consul,ou=users,dc=torproject,dc=org
gid: gitlab-consul
objectClass: top
objectClass: debianGroup
gidNumber: 2147

464 uid=gitlab-consul,ou=users,dc=torproject,dc=org
uid: gitlab-consul
objectClass: top
objectClass: debianAccount
objectClass: shadowAccount
objectClass: debianRoleAccount
gecos: gitlab-consul role account,,,,
cn: gitlab-consul role account
shadowLastChange: 17085
shadowMin: 0
shadowMax: 99999
shadowWarning: 7
userPassword: {crypt}*
loginShell: /bin/dash
gidNumber: 2147
uidNumber: 2147

      which is basically:

      gid=dip gidNumber: 2129
gid=gitlab-www gidNumber: 2143
uid=gitlab-www uidNumber: 2143
gid=gitlab-redis gidNumber: 2144
uid=gitlab-redis uidNumber: 2144
gid=gitlab-psql gidNumber: 2145
uid=gitlab-psql uidNumber: 2145
gid=registry gidNumber: 2146
uid=registry uidNumber: 2146
gid=gitlab-consul gidNumber: 2147
uid=gitlab-consul uidNumber: 2147

      this was already removed a few minutes ago (#127 (comment 2823380)):

      allowedGroups: git

      now i've also removed this from the gitlab-02 host:

          allowedGroups: dip
    allowedGroups: gitlab-consul
    allowedGroups: gitlab-redis
    allowedGroups: gitlab-www
    allowedGroups: registry

      this also involved removing ahf, dgoulet and hiro from the dip group.

    • anarcat marked the checklist item allowedGroups: gitlab-consul as completed

      marked the checklist item allowedGroups: gitlab-consul as completed

    • anarcat marked the checklist item allowedGroups: gitlab-redis as completed

      marked the checklist item allowedGroups: gitlab-redis as completed

    • anarcat marked the checklist item allowedGroups: gitlab-www as completed

      marked the checklist item allowedGroups: gitlab-www as completed

    • anarcat marked the checklist item allowedGroups: registry as completed

      marked the checklist item allowedGroups: registry as completed

    • anarcat marked the checklist item allowedGroups: dip as completed

      marked the checklist item allowedGroups: dip as completed

    • anarcat
      anarcat @anarcat ·
      Author Owner

      check for unowned files on the server: find / /var/opt /srv/gitlab-backup/ /srv/gitlab-shared/ -xdev -nouser -o -nogroup

      currently running. this already found that file:

      -rw-r----- 1 root 995 61 Feb 20  2020 /opt/gitlab/embedded/cookbooks/cache/backup/var/opt/gitlab/gitlab-workhorse/config.toml.chef-20200220160720.175045

      group 995 looks like it could be an old gid=git, but the one in /etc/group is 2149, so that predates my modifications. and besides, the timestamp is 2 years old...

      that file seems to be a cache of a backup, so it was removed.

      Edited by anarcat
    • anarcat marked the checklist item check for unowned files on the server: find / /var/opt /srv/gitlab-backup/ /srv/gitlab-shared/ -xdev -nouser -o -nogroup as completed

      marked the checklist item check for unowned files on the server: find / /var/opt /srv/gitlab-backup/ /srv/gitlab-shared/ -xdev -nouser -o -nogroup as completed

    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    • Loading
    Please register or sign in to reply