aws / aws/aws-cdk

(aws-logs): Alternative Method for setting LogRetentionPolicy

Open
#24,166 1 comment 5 reactions 0 assignees View on GitHub
@aws-cdk/aws-cloudwatch @aws-cdk/aws-logs feature-request p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the feature

I'm not certain this belongs in the CDK, but it's meant to address a somewhat counterintuitive and sometimes frustrating part of the CDK. I'm at least looking for feedback on the idea, even if is gets quickly closed.

### Use Case

Currently, my understanding of how Log Retention Policies are set by the CDK is to use a Custom Resource to proactively create a log group using an easily predetermined name and then set it's retention policy. This is because the log may not exist yet at deploy time. This makes sense, but can be frustrating and inconsistently implemented.

Other shortcomings include things like [Lambda at Edge CloudWatch log groups retention not being set](https://github.com/aws/aws-cdk/issues/12690) since they are often created in other regions.

### Proposed Solution

My shower thought idea to resolve this is to change the way Log Retention is set in the CDK, or perhaps via a newly created construct to maintain backward compatibility.

This new construct would use tagging to set a logging policy on the resource itself, then a scheduled lambda would later inspect all log groups and make adjustments to match the policy set (via tag) on the resource.

Alternately, all log retention policy settings could be stored in Parameter Store instead of using a tagging strategy. This would allow the scheduled lambda to lookup virtually any resource, even if they don't support tagging in a way that is useful.

Default log retention policies could also be established and stored in Parameter Store for any resources which are untagged.

### Other Information

This may be a better candidate for an outside construct that is not included in the CDK Construct library itself, but I wanted to start here since I see so many open issues that this could potentially resolve in one place such as:

- [(aws-rds): no way to specify CloudWatch log group retention](https://github.com/aws/aws-cdk/issues/23720)
- [RDS: Is it possible to specify LogGroup RetentionDays for ServerlessCluster?](https://github.com/aws/aws-cdk/issues/22662)
- [(lambda): Add property for log removal policy of Lambda function log groups](https://github.com/aws/aws-cdk/issues/21804)

I won't cross link them all but there appear to be a lot more.

Would probably want to consider what to do with log groups not created by CDK resources. This construct may need to steer clear of those and only pay attention to CDK resources.

### Acknowledgements

- [X] I may be able to implement this feature request
- [ ] This feature might incur a breaking change

### CDK version used

2.64.0

### Environment details (OS name and version, etc.)

All

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the CDK's existing custom-resource approach to log retention and the linked RDS and Lambda issues. Before implementation, resolve whether this belongs in CDK and choose between tagging with a scheduled Lambda and Parameter Store; done would require a defined construct design and retention behavior for CDK-created and externally created log groups.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.