key file is not used if other keys exist in ssh agent

[xinfengliu]

While debugging a k0sctl issue, I found my specified keyPath for ssh is not used and ssh connection failed.

spec:
  hosts:
  - ssh:
      address: rocky9-0
      keyPath: /home/docker/.ssh/id_ed25519
      port: 22
      user: root
...

To work around the issue, I had to manually add the key to ssh agent or unset SSH_AUTH_SOCK or clearing keys in ssh agent.

I reviewed k0sproject/rig codes and found the reason is that I have already had other ssh keys in SSH agent, and the specified ssh key file is configured as another ssh.AuthMethod in https://github.com/k0sproject/rig/blob/v0.19.0/ssh.go#L434 , so ssh.ClientConfig.Auth has two AuthMethod, both method() is publickey , golang.org/x/crypto/ssh client_auth.go only tries the first one for the same auth method (https://cs.opensource.google/go/x/crypto/+/refs/tags/v0.29.0:ssh/client_auth.go;l=101), so the specified ssh key file is never used.

[kke]

I think one problem may be if /home/docker/.ssh/id_ed25519 is the default key path, in which case rig could assume that an explicit keypath wasn't given and tries other methods 🤔 If a keypath is given, it should only try that key and nothing else.

[xinfengliu]

I tried non-default key path, but the problem remains same. I added a few debug logs, here's ~/.cache/k0sctl/k0sctl.log

time="24 Dec 24 12:05 CST" level=debug msg="retrying, attempt 8 - last error: not connected: client connect: ssh dial: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain"
time="24 Dec 24 12:05 CST" level=info msg="### SSH: [ssh] rocky9-0:22"
time="24 Dec 24 12:05 CST" level=debug msg="using SSH_AUTH_SOCK=/tmp/ssh-XXXXcro9PG/agent.1283978"
time="24 Dec 24 12:05 CST" level=debug msg="[ssh] rocky9-0:22: using all keys (2) from ssh agent because a keypath was not explicitly given"
time="24 Dec 24 12:05 CST" level=info msg="## config.Auth: [0x8f7c60]"
time="24 Dec 24 12:05 CST" level=info msg="## keyPath: /home/docker/test/mke4/id_ed25519"
time="24 Dec 24 12:05 CST" level=info msg="### keyPaths: [/home/docker/test/mke4/id_ed25519]"
time="24 Dec 24 12:05 CST" level=info msg="### config.Auth: [0x8f7c60 0x8f9120]"
time="24 Dec 24 12:05 CST" level=debug msg="[SSH] rocky9-0: failed to connect: ssh dial: ssh: handshake failed: ssh: unable to authenticate, attempted methods [none publickey], no supported methods remain"