[EKS] [request]: Allow different retention policies for different control plane log types
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
Potentially related tangentially to #975.
EKS [allows us to enable sending a number of different log types](https://docs.aws.amazon.com/eks/latest/userguide/control-plane-logs.html) from the Control Plane to Cloudwatch including audit and authenticator, however all of these log types are published as different streams under a per cluster log group. This means that all of the [different log types must have the same retention period applied to them](https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/Working-with-log-groups-and-streams.html).
We have significantly different retention requirements for different classes of logs, requiring longer retention for auth and audit logs than all other classes of logs, currently, we're unable to do this with EKS.
**Which service(s) is this request for?**
EKS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
We have a requirement to keep 1 years worth of audit and authentication logs, but aren't required to, and don't want to keep this amount of data for the other log types. We want to be cost-conscious, only keeping the logs we need, whilst also providing ourselves with the visibility into clusters and their operations that we desire.
We're currently forced into making a choice between enabling log types with shorter retention requirements and retaining them for significantly longer than required, or not enabling these log types and instead losing visibility on our clusters and their operations.
**Are you currently working around this issue?**
Currently we're not enabling the log types other than audit and authenticator as we don't want to pay the cost of retaining the other log types (controllerManager, api and scheduler) for 52 times the length of time which we want to retain them for.
Contributor guide
Research direction
Start with the linked EKS control-plane logging and CloudWatch documentation, then review the related issue #975. Done means EKS supports separate retention periods for audit, authenticator, controllerManager, api, and scheduler log types, while preserving the requested cost and visibility trade-offs.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, observability
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100