aws / aws/containers-roadmap

[EKS] [eks-pod-identity] [bug]: Setting the STS Session name in eks-pod-identity-agent

Open
#2,362 8 comments 13 reactions 0 assignees View on GitHub
EKS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

I think this is probably more of a bug report, but I cannot find the proper channels.
Version Info:
EKS: 1.29
Eks Pod ID agent: v1.2.0-eksbuild.1

We are using eks-pod-identity. It's been working great till we started using IAM based kafka.

I directly hit the eks-pod-identity pod inside a container agent via
```
AWS_CONTAINER_CREDENTIALS_FULL_URI: http://169.254.170.23/v1/credentials
AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE: /var/run/secrets/pods.eks.amazonaws.com/serviceaccount/eks-pod-identity-token │
```

I took the resulting AccessKey, SecertKey, and Token, and used them to do a `aws sts get-caller-identity`

```
{
"UserId": "AROAXYZP66II6MSBPLOUU:eks-k8s-wl-dev-engine-boo-5af5e7ac-5754-49ea-b28f-2c2a2eb95fbb",
"Account": "BLAH",
"Arn": "arn:aws:sts::BLAH:assumed-role/dev-use1-default-bookingApiPod-20240529171038526700000001/eks-k8s-wl-dev-engine-boo-5af5e7ac-5754-49ea-b28f-2c2a2eb95fbb"
}
```

The SessionName is non static, and I can't find any way to force set it. The issue comes from the MSK IAM usage. MSK doesn't allow "reauthentication". We get this error when the session name changes
```
failed authentication due to: Cannot change principals during re-authentication from IAM.arn:aws:sts::BLAH:assumed-role/prd-use1-default-bookingApiPod-20240529174141238800000002/eks-k8s-wl-prd-engine-boo-064f1ed1-2349-4774-b895-9a69ccc3eeb1: IAM.arn:aws:sts::BLAH:assumed-role/prd-use1-default-bookingApiPod-20240529174141238800000002/eks-k8s-wl-prd-engine-boo-875fdfcd-19cd-4ac3-8544-7077f94a6e39
```

Most services we've used IAM for to date don't care. The solution normally would be to set `AWS_ROLE_SESSION_NAME` when calling STS:AssumeRole. But we're not calling that, the eks-pod-identity pod is. The ContainerCredentialProvider in the SDK is just calling the `AWS_CONTAINER_CREDENTIALS_FULL_URI` URL with the contents of `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` as the Authorization header. There is no option as far as I've found to have that include a constant session name to satisfy MSK.

Thanks!

Contributor guide

Open the contributing guide

Research direction

Start by tracing the eks-pod-identity pod's credential endpoint and the ContainerCredentialProvider flow using AWS_CONTAINER_CREDENTIALS_FULL_URI and AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE; the issue names no source files or tests. Review how the agent obtains credentials through STS and how MSK IAM handles reauthentication. Done means a supported way exists to keep the STS session name stable for this workflow.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kafka
Domain
authentication, cloud
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.