aws / aws/aws-cdk

aws_eks: kubectl function shouldn't require the vpc when endpointAccess is PUBLIC_AND_PRIVATE

Open
#27,055 5 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-ec2 @aws-cdk/aws-eks @aws-cdk/aws-eks-v2-alpha bug p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

_See AWS Case ID `#13749604991` for more details._

We've been working on building out new EKS clusters based on CDK rather than native CloudFormation. As part of that, we're iterating with the integration test runner. We were struggling to understand why the `DELETE` process for the tests was taking 30+m, when we discovered that the `KubectlHandler` function is being created inside the VPC. This was surprising to us, as we specifically are not requiring that (we are not setting the [`placeClusterHandlerInVpc`](https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-eks/lib/cluster.ts#L632-L638) flag).

It turns out that the [`Cluster`](https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-eks/lib/cluster.ts#L1226) class makes the decision to place the `KubectlHandler` into the VPC at https://github.com/aws/aws-cdk/blob/main/packages/aws-cdk-lib/aws-eks/lib/cluster.ts#L1594:

```typescript
if (this.endpointAccess._config.privateAccess && privateSubnets.length !== 0) {

// when private access is enabled and the vpc has private subnets, lets connect
// the provider to the vpc so that it will work even when restricting public access.

// validate VPC properties according to: https://docs.aws.amazon.com/eks/latest/userguide/cluster-endpoint.html
if (this.vpc instanceof ec2.Vpc && !(this.vpc.dnsHostnamesEnabled && this.vpc.dnsSupportEnabled)) {
throw new Error('Private endpoint access requires the VPC to have DNS support and DNS hostnames enabled. Use `enableDnsHostnames: true` and `enableDnsSupport: true` when creating the VPC.');
}

this.kubectlPrivateSubnets = privateSubnets;

// the vpc must exist in order to properly delete the cluster (since we run `kubectl delete`).
// this ensures that.
this._clusterResource.node.addDependency(this.vpc);
}
```

While yes .. we want our clusters to be PUBLCI and PRIVATE so that within the VPC, API calls can go directly to the API without leaving the network. On the other hand, we couldn't care less about the Lambda function being inside the VPC. According to our case rep, any time a `DELETE` call is made to a Lambda function with attached ENIs, there is a 20m delay between the DELETE starting and finishing. This delay adds a huge amount of wasted time to our integration tests and basically slows everything down for little value.

### Expected Behavior

I would expect that if the cluster is `PUBLIC_AND_PRIVATE`, and I set `placeClusterHandlerInVpc: false`, that the Lambda functions would indeed NOT be placed in the VPC.

### Current Behavior

They are placed in the VPC, regardless of our `placeClusterHandlerInVpc` setting.

### Reproduction Steps

Create a cluster with PUBLIC_AND_PRIVATE set ... then try deleting it. Watch it take 20 minutes to delete.

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.94.0

### Framework Version

_No response_

### Node.js Version

18

### OS

linux

### Language

Typescript

### Language Version

_No response_

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start in packages/aws-cdk-lib/aws-eks/lib/cluster.ts around the endpointAccess logic and the placeClusterHandlerInVpc option cited in the issue. Reproduce a PUBLIC_AND_PRIVATE cluster with the option disabled and observe the integration test runner's deletion behavior. Done means the option's documented behavior is respected without breaking private endpoint access.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes, typescript
Domain
cloud, devops, infrastructure
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.