hashicorp / hashicorp/packer-plugin-amazon
ERROR log level noise: IMDSv2 token failures on every SSH operation during build
- Dominant language
- Go
- Stars
- 91
- Forks
- 141
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 3
Description
### Overview
When building an AMI from an Amazon Linux 2023 source AMI using the `amazon-ebs` builder, every SSH-based operation (shell provisioner, file upload, ansible-local) emits an ERROR-level log message:
```
Error: amazon-ebs: [ERROR] Could not get IMDSv2 token. Instance Metadata might have been disabled or this is not an EC2 instance.
```
This is **cosmetic noise** — all operations succeed, the build completes, and the AMI is created correctly. Ansible playbook shows `failed=0`. The plugin correctly retries via IMDSv2 after the IMDSv1 attempt fails.
### Root Cause
The SSH communicator (packer-plugin-sdk) or the AWS SDK's EC2 metadata client tries IMDSv1 first when connecting. If the instance (or source AMI) uses IMDSv2-only, this first attempt fails, logs an ERROR, then successfully retries with IMDSv2.
### Expected Behavior
- The initial IMDSv1 failure & retry to IMDSv2 should log at **WARN** or **DEBUG** level, not ERROR
- ERROR level implies a build failure, but the build succeeds
### Reproduction
1. Use `amazon-ebs` builder
2. Source AMI: any Amazon Linux 2023 (`base-ami-v3-*`)
3. No `metadata_options` needed in builder config
4. Run `packer build`
### Environment
- **Packer versions tested**: 1.9.2, 1.11.0, 1.14.0
- **Plugin versions tested**: 1.2.0 through 1.8.1 (all produce the same noise)
- **Source AMI**: Amazon Linux 2023 (kernel 6.1)
- **Note**: Does NOT happen with Amazon Linux 2 (kernel 4.14) — likely because the SSH sequence timing differs
### Impact
- Fills CI logs with distracting ERROR messages
- Makes it harder to identify actual build failures
- Affects every user building from AL2023 source AMIs
### Screenshot
```
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Provisioning with shell script: /tmp/packer-shell...
Error: amazon-ebs: [ERROR] Could not get IMDSv2 token. Instance Metadata might have been disabled or this is not an EC2 instance.
==> amazon-ebs: /home/ec2-user
```
The build continues fine after this message. Ansible recap: `failed=0`.
Contributor guide
Research direction
Start by reproducing the message with the amazon-ebs builder against an Amazon Linux 2023 source AMI and inspect the SSH communicator and IMDS client path in the plugin, packer-plugin-sdk, or AWS SDK. Determine which component emits the IMDSv1 failure, then verify that a successful IMDSv2 retry no longer produces an ERROR-level message while the build still completes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100