aws-samples / aws-samples/amazon-cloudwatch-container-insights
Workaround for pods not being able to access EC2 IMDS
- Dominant language
- Shell
- Stars
- 178
- Forks
- 117
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 3
Description
In my EKS cluster I have disabled the Instance Metadata Service (IMDS) v1 and set the IMDS hop count to 1, to prevent pods from accessing the IMDS. This prevents the `cloudwatch-agent` daemonset from starting, since it looks into the EC2 IMDS.
```
2020/11/12 15:56:28 I! 2020/11/12 15:56:25 E! ec2metadata is not available
2020/11/12 15:56:25 I! attempt to access ECS task metadata to determine whether I'm running in ECS.
2020/11/12 15:56:26 W! retry [0/3], unable to get http response from http://169.254.170.2/v2/metadata, error: unable to get response from http://169.254.170.2/v2/metadata, error: Get "http://169.254.170.2/v2/metadata": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
```
Is there a config item or workaround to use this agent without allowing pods to access the EC2 IMDS?
Contributor guide
Research direction
Start with the cloudwatch-agent DaemonSet and its configuration, then trace how it discovers EC2 or ECS metadata under IMDSv1-disabled, hop-count-1 settings. Reproduce the startup failure using the logged metadata endpoints and compare available configuration or deployment options; done means the agent starts and collects data without pod access to EC2 IMDS.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100