aws / aws/containers-roadmap

[EKS] [request]: Allow different retention policies for different control plane log types

Open
#1,386 1 comment 11 reactions 0 assignees View on GitHub
EKS Proposed
Dominant language
Shell
Stars
5.4k
Forks
334
PR merge metrics
No merged PRs in 30d

Description

### Community Note

* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment

**Tell us about your request**

Potentially related tangentially to #975.

EKS [allows us to enable sending a number of different log types](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html) from the Control Plane to Cloudwatch including audit and authenticator, however all of these log types are published as different streams under a per cluster log group. This means that all of the [different log types must have the same retention period applied to them](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Working-with-log-groups-and-streams.html).

We have significantly different retention requirements for different classes of logs, requiring longer retention for auth and audit logs than all other classes of logs, currently, we're unable to do this with EKS.

**Which service(s) is this request for?**
EKS

**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**

We have a requirement to keep 1 years worth of audit and authentication logs, but aren't required to, and don't want to keep this amount of data for the other log types. We want to be cost-conscious, only keeping the logs we need, whilst also providing ourselves with the visibility into clusters and their operations that we desire.

We're currently forced into making a choice between enabling log types with shorter retention requirements and retaining them for significantly longer than required, or not enabling these log types and instead losing visibility on our clusters and their operations.

**Are you currently working around this issue?**

Currently we're not enabling the log types other than audit and authenticator as we don't want to pay the cost of retaining the other log types (controllerManager, api and scheduler) for 52 times the length of time which we want to retain them for.

Contributor guide

Open the contributing guide

Research direction

Start with the linked EKS control-plane logging and CloudWatch documentation, then review the related issue #975. Done means EKS supports separate retention periods for audit, authenticator, controllerManager, api, and scheduler log types, while preserving the requested cost and visibility trade-offs.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, kubernetes
Domain
cloud, observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.