aws / aws/aws-cdk

aws-eks: Configure logging for cluster resource provider

Open
#31,745 2 comments 0 reactions 0 assignees View on GitHub
@aws-cdk/aws-eks feature-request p3
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the feature

We'd like to be able to configure logging settings for the custom resource provider used to deploy EKS clusters. We'd like to be able to set the log groups for the lambdas and the provider. We need to be able to do this via JSii.

### Use Case

We'd like to be able to deploy EKS clusters in a context where the log retention is fixed for the AWS account. In this context, any stack will fail to deploy if it attempts to create a CloudWatch log group without explicitly setting infinite log retention, so that the account configuration will take precedence.

We also have a requirement that we need to be able to remove the EKS cluster from an existing deployment, and then later add it back with the same configuration. We currently do this with an optional nested stack that is removed or added based on configuration. We'd like to be able to retain the log groups when we remove the optional stack, and then have it reattach to the same log groups when we add it back in. We're trying to achieve this by declaring the log groups in a separate stack. If we don't do that, when you remove the optional stack it keeps the log groups around, then when you add it back in it fails to deploy because the log groups already exist.

This includes any log groups associated with a resource provider, such as implicit log groups for a lambda or a waiter state machine for a custom resource.

### Proposed Solution

It looks like the relevant resources are deployed in a nested stack ClusterResourceProvider (see [cluster-resource-provider.ts](https://github.com/aws/aws-cdk/blob/f4f8abcb2a6df6a26b289b49b7738efce78b2936/packages/aws-cdk-lib/aws-eks/lib/cluster-resource-provider.ts)). This is discovered by ClusterResource and FargateProfile, and if that nested stack already exists in the context, those resources will use the preconfigured instance. It's not currently possible to set log groups for that nested stack, or to create an instance of it via JSii.

Any way to configure log groups for that nested stack via JSii would solve the problem, either against the cluster or against a separate entity that just configures the nested stack.

We'll need to be able to set log groups for the two lambdas deployed under ClusterResourceProvider, the lambda deployed under the Provider there, and the waiter state machine that's also deployed under the Provider. ProviderProps has settings waiterStateMachineLogOptions, logRetention and logGroup. Those could be set from ClusterResourceProvider.

### Other Information

This causes the following issues, which we will be unable to solve until this is resolved:
- https://github.com/gchq/sleeper/issues/3451
- https://github.com/gchq/sleeper/issues/3480

### Acknowledgements

- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

v2.162.1

### Environment details (OS name and version, etc.)

Ubuntu 24.04

Contributor guide

Open the contributing guide

Research direction

Start with packages/aws-cdk-lib/aws-eks/lib/cluster-resource-provider.ts and trace how ClusterResource and FargateProfile discover or create the nested ClusterResourceProvider. Review ProviderProps, especially waiterStateMachineLogOptions, logRetention, and logGroup, then determine how JSii can expose configuration for the two lambdas, provider lambda, and waiter state machine. Done means those log groups and retention settings can be configured and reused through the cluster or a separate entity.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.