aws / aws/aws-cdk

(pipelines): Allow configuring log retention period for CdkPipeline

Open
#13,250 9 comments 24 reactions 0 assignees View on GitHub
@aws-cdk/pipelines effort/medium feature-request p1
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

Cloudwatch logs are costly resources and CdkPipeline by default creates separate log groups for each build action. This means in practice building 30 lambda functions creates 30 log groups each of which is updated on each build. All other stages and actions also create their own logs groups.

Further compounding the problem is the default retention period for the logs which is Infinite (or 2 years). This can very very quickly increase the cost of the infrastructure.

### Use Case

Frequent deployment creates a lot of logs that retained infinitely adding up to the total cost of infrastructure.

### Proposed Solution

Maybe another parameter in the [CdkPipeline](https://docs.aws.amazon.com/cdk/api/latest/docs/@aws-cdk_pipelines.CdkPipeline.html) construct for passing in a custom log group [like in api-gateway](https://docs.aws.amazon.com/cdk/api/latest/docs/aws-apigateway-readme.html#access-logging) might work. However that may not be a good level of abstraction as people might want to keep build logs separate from deploy logs.

Another approach might be to create a parameter for retention period that is applied to all log groups created under the CdkPipeline construct.

Workaround is probably creating your own CodePipeline which would defeat the purpose of having CdkPipeline.

This might also be considered for GA release discussion for CdkPipeline #10872

### Other

* [x] :wave: I may be able to implement this feature request
* [ ] :warning: This feature might incur a breaking change

---

This is a :rocket: Feature Request

Contributor guide

Open the contributing guide

Research direction

Start with the CdkPipeline construct and the CloudWatch log groups it creates, then compare the proposed custom log group and retention-period approaches. Define how retention should apply across build, deploy, and other pipeline actions, and verify that the chosen configuration covers the relevant log groups without breaking existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.