aws / aws/amazon-eks-pod-identity-webhook
Pod Identity Association also subject to cache race condition?
- Dominant language
- Go
- Stars
- 691
- Forks
- 202
- Avg merge
- 4h 38m
- Merged PRs (30d)
- 1
Description
**What happened**:
Similar to #174 but specific to pod identity associations, we're observing the expected `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` env var is absent when a service account and pod are created within a short window. Typically we'll experience something like this:
- Programmatically create a pod identity association, service account annotated with IAM role, and pod in short succession
- The pod comes up, but AWS operations error with `An error occurred (InvalidIdentityToken) when calling the AssumeRoleWithWebIdentity operation: No OpenIDConnect provider found in your account for https://oidc.eks...`
- Examining the pod env, note that `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` is missing but `AWS_WEB_IDENTITY_TOKEN_FILE` is set
- Restart pod
- Note that `AWS_WEB_IDENTITY_TOKEN_FILE` is now replaced with `AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE` and pod operates as expected.
**What you expected to happen**:
If all the prerequisites are satisfied, pods should get the correct pod identity association mutation regardless of timing.
**How to reproduce it (as minimally and precisely as possible)**:
1. Create an IAM role with the correct pod identity association trust policy:
```yaml
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": "pods.eks.amazonaws.com"
},
"Action": [
"sts:TagSession",
"sts:AssumeRole"
],
}
]
}
```
2. Create an EKS cluster, enabling the EKS Pod Identity Agent add-on.
3. Run `aws eks update-kubeconfig --name my-cluster`
4. Run:
```bash
$ aws eks create-pod-identity-association \
--cluster-name my-cluster \
--namespace default \
--service-account test-sa \
--role-arn arn:aws:iam::111111111111:role/test-role && \
sleep 0.75s && \
kubectl apply -f - <
Contributor guide
Research direction
Start by reproducing the timing-sensitive sequence with aws eks create-pod-identity-association followed by kubectl apply, then inspect pkg/containercredentials/config.go and its FileConfig entry point. Compare the behavior with issues #174, #236, and #252; done means newly created pods consistently receive AWS_CONTAINER_AUTHORIZATION_TOKEN_FILE when the prerequisites are satisfied.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, go, kubernetes
- Domain
- authentication, cloud, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100