# Symptom While tackling #18184, i removed @groente's SSH key from hiera and manually ran `puppet agent -t` on `apt.lizard`, which resulted in errors for (all?) `ssh_authorized_key` resources: ``` root@apt:~# puppet agent -t Info: Using environment 'production' Info: Retrieving pluginfacts Info: Retrieving plugin Info: Loading facts Info: Caching catalog for apt.lizard Info: Applying configuration version '1740075967' Notice: /Stage[main]/Tails::Profile::Reprepro::Snapshots::Tagged/Package[hardlink]/ensure: created (corrective) Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-anonym-reprepro-time-based-snapshots]/Ssh_authorized_key[anonym_lizard-reprepro-time-based-snapshots]/ensure: created (corrective) Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-anonym-reprepro-time-based-snapshots]/Ssh_authorized_key[anonym_lizard-reprepro-time-based-snapshots]: Could not evaluate: Path is nil Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-anonym-reprepro-time-based-snapshots]/Ssh_authorized_key[anonym_personal-reprepro-time-based-snapshots]/ensure: created (corrective) Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-anonym-reprepro-time-based-snapshots]/Ssh_authorized_key[anonym_personal-reprepro-time-based-snapshots]: Could not evaluate: Path is nil Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-boyska-reprepro-time-based-snapshots]/Ssh_authorized_key[boyska_personal-reprepro-time-based-snapshots]/ensure: created (corrective) Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-boyska-reprepro-time-based-snapshots]/Ssh_authorized_key[boyska_personal-reprepro-time-based-snapshots]: Could not evaluate: Path is nil Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_backups-reprepro-time-based-snapshots]/ensure: created (corrective) Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_backups-reprepro-time-based-snapshots]: Could not evaluate: Path is nil Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_lizard-reprepro-time-based-snapshots]/ensure: created (corrective) Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_lizard-reprepro-time-based-snapshots]: Could not evaluate: Path is nil Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_personal-reprepro-time-based-snapshots]/ensure: created (corrective) Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_personal-reprepro-time-based-snapshots]: Could not evaluate: Path is nil Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_iguana-reprepro-time-based-snapshots]/ensure: created (corrective) Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_iguana-reprepro-time-based-snapshots]: Could not evaluate: Path is nil Info: Stage[main]: Unscheduling all events on Stage[main] Notice: Applied catalog in 17.73 seconds ``` # Investigation When i add `-d`, the debug log says something that is untrue: ``` Debug: The required user is not yet present on the system ``` The `reprepro-time-based-snapshots` does exist (and has always existed) in the system: ```console root@apt:~# id reprepro-time-based-snapshots uid=998(reprepro-time-based-snapshots) gid=998(reprepro-time-based-snapshots) groups=998(reprepro-time-based-snapshots) ``` I added debugging lines to the `ssh_authorized_key` type definition: ```diff diff --git a/lib/puppet/type/ssh_authorized_key.rb b/lib/puppet/type/ssh_authorized_key.rb index 701dfe5..df35036 100644 --- a/lib/puppet/type/ssh_authorized_key.rb +++ b/lib/puppet/type/ssh_authorized_key.rb @@ -113,8 +113,10 @@ module Puppet begin File.expand_path("~#{resource[:user]}/.ssh/authorized_keys") - rescue + rescue => e Puppet.debug 'The required user is not yet present on the system' + Puppet.debug "Rescued: #{e.inspect}" + Puppet.debug "Path: ~#{resource[:user]}/.ssh/authorized_keys" nil end end ``` We can see that the actual error is `#<RuntimeError: can't set length of shared string>` when running the `File.expand_path(...)` line: ``` Notice: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_iguana-reprepro-time-based-snapshots]/ensure: created (corrective) Debug: The required user is not yet present on the system Debug: Rescued: #<RuntimeError: can't set length of shared string> Debug: Path: ~reprepro-time-based-snapshots/.ssh/authorized_keys Error: /Stage[main]/Tails::Profile::Reprepro/Rbac::Ssh[foundations-team-members-to-reprepro-time-based-snapshots]/Rbac::Sshto[foundations-team-members-intrigeri-reprepro-time-based-snapshots]/Ssh_authorized_key[intrigeri_iguana-reprepro-time-based-snapshots]: Could not evaluate: Path is nil ``` Also, the path is correct and actually does exists (and has always existed): ``` root@apt:~# ls -alh ~reprepro-time-based-snapshots/.ssh/authorized_keys -rw------- 1 reprepro-time-based-snapshots reprepro-time-based-snapshots 5.7K Nov 13 2023 /srv/apt-snapshots/time-based/.ssh/authorized_keys ``` /cc @anarcat @groente @lavamind @lelutin, do any of you have a clue?