aws_eks: kubectl function shouldn't require the vpc when endpointAccess is PUBLIC_AND_PRIVATE
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 83
Description
### Describe the bug
_See AWS Case ID `#13749604991` for more details._
We've been working on building out new EKS clusters based on CDK rather than native CloudFormation. As part of that, we're iterating with the integration test runner. We were struggling to understand why the `DELETE` process for the tests was taking 30+m, when we discovered that the `KubectlHandler` function is being created inside the VPC. This was surprising to us, as we specifically are not requiring that (we are not setting the [`placeClusterHandlerInVpc`](https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-eks/lib/cluster.ts#L632-L638) flag).
It turns out that the [`Cluster`](https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-eks/lib/cluster.ts#L1226) class makes the decision to place the `KubectlHandler` into the VPC at https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-eks/lib/cluster.ts#L1594:
```typescript
if (this.endpointAccess._config.privateAccess && privateSubnets.length !== 0) {
// when private access is enabled and the vpc has private subnets, lets connect
// the provider to the vpc so that it will work even when restricting public access.
// validate VPC properties according to: https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html
if (this.vpc instanceof ec2.Vpc && !(this.vpc.dnsHostnamesEnabled && this.vpc.dnsSupportEnabled)) {
throw new Error('Private endpoint access requires the VPC to have DNS support and DNS hostnames enabled. Use `enableDnsHostnames: true` and `enableDnsSupport: true` when creating the VPC.');
}
this.kubectlPrivateSubnets = privateSubnets;
// the vpc must exist in order to properly delete the cluster (since we run `kubectl delete`).
// this ensures that.
this._clusterResource.node.addDependency(this.vpc);
}
```
While yes .. we want our clusters to be PUBLCI and PRIVATE so that within the VPC, API calls can go directly to the API without leaving the network. On the other hand, we couldn't care less about the Lambda function being inside the VPC. According to our case rep, any time a `DELETE` call is made to a Lambda function with attached ENIs, there is a 20m delay between the DELETE starting and finishing. This delay adds a huge amount of wasted time to our integration tests and basically slows everything down for little value.
### Expected Behavior
I would expect that if the cluster is `PUBLIC_AND_PRIVATE`, and I set `placeClusterHandlerInVpc: false`, that the Lambda functions would indeed NOT be placed in the VPC.
### Current Behavior
They are placed in the VPC, regardless of our `placeClusterHandlerInVpc` setting.
### Reproduction Steps
Create a cluster with PUBLIC_AND_PRIVATE set ... then try deleting it. Watch it take 20 minutes to delete.
### Possible Solution
_No response_
### Additional Information/Context
_No response_
### CDK CLI Version
2.94.0
### Framework Version
_No response_
### Node.js Version
18
### OS
linux
### Language
Typescript
### Language Version
_No response_
### Other information
_No response_
Contributor guide
Research direction
Start in packages/aws-cdk-lib/aws-eks/lib/cluster.ts around the endpointAccess logic and the placeClusterHandlerInVpc option cited in the issue. Reproduce a PUBLIC_AND_PRIVATE cluster with the option disabled and observe the integration test runner's deletion behavior. Done means the option's documented behavior is respected without breaking private endpoint access.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes, typescript
- Domain
- cloud, devops, infrastructure
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100