(aws-eks): `aws-auth` ConfigMap is still being replaced
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### What is the problem?
#7981 was not fixed by #8447 – The `aws-auth` ConfigMap is still being replaced, rather than appended to, any time the `Cluster.awsAuth` getter is invoked, which happens whenever you add a Fargate Profile to a cluster, forcing all Fargate Profiles to originate from the same instance of `Cluster` from `@aws-cdk/aws-eks/cluster.ts`.
In [the AwsAuth constructor](https://github.com/aws/aws-cdk/blob/04cf28ed7f9da1608d6d53c117d7b535bb1b299d/packages/%40aws-cdk/aws-eks/lib/aws-auth.ts#:~:text=*/-,export%20class%20AwsAuth%20extends%20Construct%20%7B,%7D,-/**), the following manifest is added added to the cluster:
```ts
new KubernetesResource(this, 'manifest', {
cluster: props.cluster,
manifest: [
{
apiVersion: 'v1',
kind: 'ConfigMap',
metadata: {
name: 'aws-auth',
namespace: 'kube-system',
},
data: {
mapRoles: this.synthesizeMapRoles(),
mapUsers: this.synthesizeMapUsers(),
mapAccounts: this.synthesizeMapAccounts(),
},
},
],
});
```
This constructor is invoked whenever the `Cluster.awsAuth` getter is invoked, which happens any time you invoke `Cluster.awsAuth.addRoleMapping`, which happens any time you invoke `Cluster.addFargateProfile`.
What this means is that if any software changes the `mapRoles` of the `aws-auth` ConfigMap without invoking `Cluster.awsAuth.addRoleMapping` on that particular instance of `Cluster`, those changes will be erased upon the next addition of a Fargate Profile.
In other words, upon each instantiation of `Cluster` where the `Cluster.awsAuth` getter is invoked, all ARNs of all `PodExecutionRole`s added manually with `kubectl`, other CDK apps, other IaC tools, an L1 Construct like `CfnFargateProfile`, etc. will be removed from the `aws-auth` ConfigMap, causing this error below, and preventing any new pods from spinning up.
> fargate-scheduler Misconfigured Fargate Profile: fargate profile blocked for new launches due to: Pod execution role is not found in auth config or does not have all required permissions for launching fargate pods.
### Reproduction Steps
See #7981
### What did you expect to happen?
See #7981
### What actually happened?
See #7981
### CDK CLI Version
1.85.0
### Framework Version
1.85.0
### Node.js Version
v14.17.1
### OS
macOS
### Language
Typescript
### Language Version
TypeScript (4.0.3)
### Other information
#### EKS version
1.19
#### Fix suggestion
Instead of applying a manifest to the cluster, an `AwsAuth` instance should create the `aws-auth` manifest if and only if the ConfigMap doesn't already exist in the cluster.
When `aws-auth` already exists, the result should be [a `kubectl patch` of the existing `aws-auth` ConfigMap](https://www.eksworkshop.com/intermediate/220_codepipeline/configmap/) (pictured below):

Contributor guide
Research direction
Start by reading packages/@aws-cdk/aws-eks/lib/aws-auth.ts and cluster.ts, especially the AwsAuth constructor and the Cluster.awsAuth and addFargateProfile entry points. Review issue #7981 and its reproduction. Done means an existing aws-auth ConfigMap retains externally added roles while CDK additions are applied without replacing it.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes, typescript
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100