Bare metal ssh key changes on reboot
- Dominant language
- Go
- Stars
- 2.1k
- Forks
- 328
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 9
Description
**What happened**:
When I SSH into a system I provisioned on bare metal I can see the SSH key fingerprint
```
ssh ec2-user@10.10.1.53
The authenticity of host '10.10.1.53 (10.10.1.53)' can't be established.
ECDSA key fingerprint is SHA256:bA3B01cj2lzzmmGFBETC7rQcok7gYgIzMtWXqPpVpTI.
Are you sure you want to continue connecting (yes/no/[fingerprint])?
```
When I reboot the system (not reprovisioning) the SSH key changes and I get a warning showing a new key fingerprint
```
ssh ec2-user@10.10.1.53
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
@ WARNING: REMOTE HOST IDENTIFICATION HAS CHANGED! @
@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@@
IT IS POSSIBLE THAT SOMEONE IS DOING SOMETHING NASTY!
Someone could be eavesdropping on you right now (man-in-the-middle attack)!
It is also possible that a host key has just been changed.
The fingerprint for the ECDSA key sent by the remote host is
SHA256:eyGntJB+/F3lHeHQfZWGVFVVoTI9ADrgCjB01K+kevo.
Please contact your system administrator.
```
**What you expected to happen**:
The SSH key should stay the same when rebooting and only change when re-provisioning the OS.
**How to reproduce it (as minimally and precisely as possible)**:
* Provision a host
* SSH into it (accept the key)
* reboot the host
* SSH into it again
**Anything else we need to know?**:
Listing the keys on the system before and after reboot didn't appear to be different but I suspect something in cloud-init is causing a new key to be created.
First boot
```
ssh-keyscan localhost
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
localhost ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLZttqagIsW8Sy8mvuQG9UE5i4Hsn5F928PYMOCmNcGTEidp/iuqLgodQXFGPdL47b4LAYMLbcYPD
r70KvTcgCKbPOtl2QqZB4KsdymFpmQA5szQOF78J5GjAEBZGL3RE05IWm9ni5VBJFLDjknE0yDipgNylWVWJi1CTtLXJNMqFR6uChQ5RQghFxpdBc9klwZiWm4FbH
JaysxjX4LGeWi6WR5z50PMIxGAhNSrS7qQlQ0UeimFWqUdBpyChrO2Jwe6yhl91Pzx4z20yGESaQYXZ5V6AK6MKtotvrWvKQAtFPoyQr991+vAh91xienepjMWgCF
kUKmWFxtFcqFRxoHmr7n/tgayfNcCBzABSmAORvCJ/07ixY2QNBVf7CY8tBnashXby5otgVPJ0aR1XyJQiAZk7ZvXDGddWXrSmt7HIYueGApu6z5IgmVlT3zir1iR
ycanbAyHHbwTI+KpkuZzQGiPlEyOlX4JXUGnY97H9P/NERFnDff/XJTwH4d+U=
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
localhost ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD0lXXSkH4UsvsZMvqQrfH5Er2+TT6IRe0aCgVCPRt8
F/pneuRpjDVT1m3tt4nkASq7SU2T2OED5g2tkXYM6xgE=
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
localhost ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO8ZbQ1TKjknr9okCvzu6Y/1br5RiqiRWFhsPri6qzN6
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
```
after reboot
```
$ ssh-keyscan localhost
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
localhost ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDLZttqagIsW8Sy8mvuQG9UE5i4Hsn5F928PYMOCmNcGTEidp/iuqLgodQXFGPdL47b4LAYMLbcYPD
r70KvTcgCKbPOtl2QqZB4KsdymFpmQA5szQOF78J5GjAEBZGL3RE05IWm9ni5VBJFLDjknE0yDipgNylWVWJi1CTtLXJNMqFR6uChQ5RQghFxpdBc9klwZiWm4FbH
JaysxjX4LGeWi6WR5z50PMIxGAhNSrS7qQlQ0UeimFWqUdBpyChrO2Jwe6yhl91Pzx4z20yGESaQYXZ5V6AK6MKtotvrWvKQAtFPoyQr991+vAh91xienepjMWgCF
kUKmWFxtFcqFRxoHmr7n/tgayfNcCBzABSmAORvCJ/07ixY2QNBVf7CY8tBnashXby5otgVPJ0aR1XyJQiAZk7ZvXDGddWXrSmt7HIYueGApu6z5IgmVlT3zir1iR
ycanbAyHHbwTI+KpkuZzQGiPlEyOlX4JXUGnY97H9P/NERFnDff/XJTwH4d+U=
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
localhost ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBD0lXXSkH4UsvsZMvqQrfH5Er2+TT6IRe0aCgVCPRt8
F/pneuRpjDVT1m3tt4nkASq7SU2T2OED5g2tkXYM6xgE=
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
localhost ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIO8ZbQ1TKjknr9okCvzu6Y/1br5RiqiRWFhsPri6qzN6
# localhost:22 SSH-2.0-OpenSSH_8.2p1 Ubuntu-4ubuntu0.5
```
**Environment**:
- EKS Anywhere Release: 0.10.0
Contributor guide
Research direction
Start by reproducing the bare-metal provisioning, SSH connection, and reboot sequence described in the issue, then inspect the cloud-init host-key handling suspected by the reporter. Compare the SSH host keys before and after reboot with ssh-keyscan; done means the keys remain stable across reboot and change only after reprovisioning.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, ubuntu
- Domain
- infrastructure, operating-systems, security
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100