aws-samples / aws-samples/amazon-cloudwatch-container-insights
AWS CloudWatch Agent failing on OCP 4.6 cluster
- Dominant language
- Shell
- Stars
- 178
- Forks
- 117
- Avg merge
- 1d 10h
- Merged PRs (30d)
- 3
Description
I am trying to set up aws CloudWatch Agent to Collect ocp 4.6 Cluster Metrics using this link https://docs.amazonaws.cn/en_us/AmazonCloudWatch/latest/monitoring/Container-Insights-setup-metrics.html
but the pods are failing with following errors:
```
[ec2-user@ip-10-0-2-100 ~]$ oc logs cloudwatch-agent-57htk
2021/02/02 21:17:21 I! 2021/02/02 21:17:18 E! ec2metadata is not available
2021/02/02 21:17:18 I! attempt to access ECS task metadata to determine whether I'm running in ECS.
2021/02/02 21:17:19 W! retry [0/3], unable to get http response from http://169.254.170.2/v2/metadata, error: unable to get response from http://169.254.170.2/v2/metadata, error: Get "http://169.254.170.2/v2/metadata": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2021/02/02 21:17:20 W! retry [1/3], unable to get http response from http://169.254.170.2/v2/metadata, error: unable to get response from http://169.254.170.2/v2/metadata, error: Get "http://169.254.170.2/v2/metadata": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2021/02/02 21:17:21 W! retry [2/3], unable to get http response from http://169.254.170.2/v2/metadata, error: unable to get response from http://169.254.170.2/v2/metadata, error: Get "http://169.254.170.2/v2/metadata": context deadline exceeded (Client.Timeout exceeded while awaiting headers)
2021/02/02 21:17:21 I! access ECS task metadata fail with response unable to get response from http://169.254.170.2/v2/metadata, error: Get "http://169.254.170.2/v2/metadata": context deadline exceeded (Client.Timeout exceeded while awaiting headers), assuming I'm not running in ECS.
I! Detected the instance is OnPrem
2021/02/02 21:17:21 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/bin/default_linux_config.json ...
/opt/aws/amazon-cloudwatch-agent/bin/default_linux_config.json does not exist or cannot read. Skipping it.
2021/02/02 21:17:21 Reading json config file path: /etc/cwagentconfig/..2021_02_02_21_14_55.157039078/cwagentconfig.json ...
2021/02/02 21:17:21 unable to scan config dir /etc/cwagentconfig with error: unable to parse json, error: invalid character '\n' in string literal
No json config files found, please provide config, exit now
2021/02/02 21:17:21 I! Return exit error: exit code=99
2021/02/02 21:17:21 I! there is no json configuration when running translator
```
Contributor guide
Research direction
Start with the AWS Container Insights setup link and reproduce the failure from the cloudwatch-agent pod logs. Inspect the /etc/cwagentconfig/.../cwagentconfig.json path and the reported JSON parse error; done means the agent pods start successfully and collect OCP 4.6 cluster metrics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, devops, observability-sre
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100