consul not using iam role linked to service account on EKS
- Dominant language
- Go
- Stars
- 30.1k
- Forks
- 4.6k
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 43
Description
#### Overview of the Issue
I'm currently running EKS and using the official helm chart to deploy a consul cluster that would join an external cluster using the cloud autodiscover. I followed the AWS doc to modify the serviceaccount in order for it to map to an IAM role. However the consul ( version 1.8.0 ) ignores these credentials and tries to access the node metadata ( which fails because I set up the recommendation in order to block access to node instance profile from pods ) .
Can someone help me around that please ?
* the extra config from the pod
```
/ # cat /consul/userconfig/consul/config.json
{"primary_datacenter":"eu-west-3-int-test","retry_join_wan":["provider=aws tag_key=Name tag_value=int-infra-consul-test-server region=eu-west-3"]}
/ #
```
* The env vars seem ok from the pod
```
/ # env |grep AWS
AWS_ROLE_ARN=arn:aws:iam::XXXXXX:role/YYYYYY
AWS_WEB_IDENTITY_TOKEN_FILE=/var/run/secrets/eks.amazonaws.com/serviceaccount/token
```
* The error logs from the pod
```
2020-08-18T18:56:45.264Z [INFO] agent: discover-aws: Address type is not supported. Valid values are {private_v4,public_v4,public_v6}. Falling back to 'private_v4': cluster=WAN
2020-08-18T18:56:45.264Z [INFO] agent: discover-aws: Region is eu-west-3: cluster=WAN
2020-08-18T18:56:45.318Z [INFO] agent: discover-aws: Filter instances with Name=int-infra-consul-test-server: cluster=WAN
2020-08-18T18:57:26.042Z [ERROR] agent: Cannot discover address: cluster=WAN address="provider=aws tag_key=Name tag_value=int-infra-consul-test-server region=eu-west-3" error="discover-aws: DescribeInstancesInput failed: NoCredentialProviders: no valid providers in chain. Deprecated.
For verbose messaging see aws.Config.CredentialsChainVerboseErrors"
2020-08-18T18:57:26.042Z [WARN] agent: Join cluster failed, will retry: cluster=WAN retry_interval=30s error="No servers to join"
```
* After installing AWS CLI on the pod the vars seem to work ( the assumed role matched the AWS_ROLE_ARN)
```
/ # aws sts get-caller-identity
{
"Account": "XXXXXXXXXXXXXXX",
"UserId": "ZZZZZZZZZZZZZZZ:botocore-session-1597777922",
"Arn": "arn:aws:sts::XXXXXXXXXX:assumed-role/YYYYYYYYY/botocore-session-1597777922"
```
### Operating system and Environment details
- EKS: v1.17.9-eks-4c6976
- CONSUL: 1.18.0
Contributor guide
Assessment
This issue has not been assessed yet.