hashicorp / hashicorp/packer-plugin-amazon

ERROR log level noise: IMDSv2 token failures on every SSH operation during build

Open
#686 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
91
Forks
141
Avg merge
2d 18h
Merged PRs (30d)
3

Description

### Overview

When building an AMI from an Amazon Linux 2023 source AMI using the `amazon-ebs` builder, every SSH-based operation (shell provisioner, file upload, ansible-local) emits an ERROR-level log message:

```
Error: amazon-ebs: [ERROR] Could not get IMDSv2 token. Instance Metadata might have been disabled or this is not an EC2 instance.
```

This is **cosmetic noise** — all operations succeed, the build completes, and the AMI is created correctly. Ansible playbook shows `failed=0`. The plugin correctly retries via IMDSv2 after the IMDSv1 attempt fails.

### Root Cause

The SSH communicator (packer-plugin-sdk) or the AWS SDK's EC2 metadata client tries IMDSv1 first when connecting. If the instance (or source AMI) uses IMDSv2-only, this first attempt fails, logs an ERROR, then successfully retries with IMDSv2.

### Expected Behavior

- The initial IMDSv1 failure & retry to IMDSv2 should log at **WARN** or **DEBUG** level, not ERROR
- ERROR level implies a build failure, but the build succeeds

### Reproduction

1. Use `amazon-ebs` builder
2. Source AMI: any Amazon Linux 2023 (`base-ami-v3-*`)
3. No `metadata_options` needed in builder config
4. Run `packer build`

### Environment

- **Packer versions tested**: 1.9.2, 1.11.0, 1.14.0
- **Plugin versions tested**: 1.2.0 through 1.8.1 (all produce the same noise)
- **Source AMI**: Amazon Linux 2023 (kernel 6.1)
- **Note**: Does NOT happen with Amazon Linux 2 (kernel 4.14) — likely because the SSH sequence timing differs

### Impact

- Fills CI logs with distracting ERROR messages
- Makes it harder to identify actual build failures
- Affects every user building from AL2023 source AMIs

### Screenshot

```
==> amazon-ebs: Connected to SSH!
==> amazon-ebs: Provisioning with shell script: /tmp/packer-shell...
Error: amazon-ebs: [ERROR] Could not get IMDSv2 token. Instance Metadata might have been disabled or this is not an EC2 instance.
==> amazon-ebs: /home/ec2-user
```

The build continues fine after this message. Ansible recap: `failed=0`.

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the message with the amazon-ebs builder against an Amazon Linux 2023 source AMI and inspect the SSH communicator and IMDS client path in the plugin, packer-plugin-sdk, or AWS SDK. Determine which component emits the IMDSv1 failure, then verify that a successful IMDSv2 retry no longer produces an ERROR-level message while the build still completes.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, go
Domain
cloud, devops
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.