aws / aws/aws-cdk

SubscriptionFilter: Rate exceeded when large number of functions present in stack for `DescribeSubscriptionFilters ` API

Open
#34,679 3 comments 2 reactions 0 assignees View on GitHub
@aws-cdk/aws-lambda @aws-cdk/aws-logs bug effort/medium p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

We are receiving rate limit exceeded errors and CDK deployment failures for cloud watch subscription in a stack with ~70 lambdas.

```ts
nodeJsFunctions.forEach((lambdaFunction) => {
const lambdaFunctionName = lambdaFunction.node.id;

new SubscriptionFilter(this, `${lambdaFunctionName}-Cloud-Watch-subscription`, {
logGroup: lambdaFunction.logGroup,
destination: new KinesisDestination(stream),
filterPattern: FilterPattern.allEvents(),
filterName: `${lambdaFunction.functionName}-cdk-subscription-${lambdaFunctionName}`,
});
```

We will eventually get:

> Ivr3ApplicationStack | 6:26:15 PM | UPDATE_FAILED | AWS::Logs::SubscriptionFilter | ivr3-getShipperAudioName-Cloud-Watch-subscription (ivr3getShipperAudioNameCloudWatchsubscription12F1FD61) Resource handler returned message: "Rate exceeded (Service: CloudWatchLogs, Status Code: 400, Request ID: 94da51db-9284-46c6-9356-7a4cc791b2c6) (SDK Attempt Count: 5)" (RequestToken: 9df95c0e-55f5-03ec-1192-3fc736e110e9, HandlerErrorCode: GeneralServiceException)

CDK exceeds rate limits on `DescribeSubscriptionFilters `

![Image](https://github.com/user-attachments/assets/ef981dac-5bed-413c-98a6-e3d3fe0fff7d)

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Last Known Working CDK Library Version

Unknown

### Expected Behavior

CDK observes AWS account rate limits and throttles API access

### Current Behavior

CDK does not observe AWS account rate limits, and blows past them, causing deployment failures

### Reproduction Steps

Deploy a number of lambdas with cloudwatch SubscriptionFilter's and observe rate limits being reached

### Possible Solution

CDK observes AWS account rate limits and throttles API access

### Additional Information/Context

_No response_

### AWS CDK Library version (aws-cdk-lib)

2.182.0

### AWS CDK CLI version

2.1003.0

### Node.js Version

22

### OS

Linux

### Language

TypeScript

### Language Version

5.8.2

### Other information

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by tracing the SubscriptionFilter deployment path and its calls to the CloudWatch Logs DescribeSubscriptionFilters API. Reproduce the failure with a stack containing roughly 70 Lambda functions and observe the rate-limit behavior. Done means deployments avoid rate-exceeded failures by respecting the account API limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.