[S3] Support Multiple S3 Object Notification Event Types for a single Event Notification Configuration
- Dominant language
- TypeScript
- Stars
- 12.9k
- Forks
- 4.6k
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 74
Description
The `S3.PutBucketNotificationConfiguration` API supports an _Array of Strings_ for the `EventType` field, however currently the AWS CDK only allows passing a single [EventType enum](https://github.com/aws/aws-cdk/blob/f0c76ac1f930fcbe7a2610e7aeeb4a46721516e1/packages/@aws-cdk/aws-s3/lib/bucket.ts#L1687-L1790) value into the `@aws-cdk/aws-s3/bucket#addEventNotification()` function.
The `addEventNotification()` function should support an Array of S3 Event Types (`EventType[]`) to pass to the S3 PutBucketNotificationConfiguration API.
### Use Case
To support multiple event types for a single S3 Bucket Notification Configuration.
I need to have an SQS Queue or SNS Topic which receives both Object Created events as well as Object Deleted Events.
### Proposed Solution
Support an (optional?) array of events:
`packages/@aws-cdk/aws-s3/lib/notifications-resource/notifications-resource.ts#L51-L60`:
https://github.com/aws/aws-cdk/blob/90de605a0f9c5487c302e34db5f89dcd58e3b445/packages/@aws-cdk/aws-s3/lib/notifications-resource/notifications-resource.ts#L51-L60
`packages/@aws-cdk/aws-s3/lib/bucket.ts#L1319-L1321`:
https://github.com/aws/aws-cdk/blob/f0c76ac1f930fcbe7a2610e7aeeb4a46721516e1/packages/@aws-cdk/aws-s3/lib/bucket.ts#L1319-L1321
The signature from `notifications-resource.ts` could support backwards compatibility:
```typescript
public addNotification(events: EventType | EventType[], target: IBucketNotificationDestination, ...filters: NotificationKeyFilter[]) {
// ... snip ...
// If `events` isn't an Array, add the single event to an array to support backwards compatibility.
const commonConfig: CommonConfiguration = {
Events: Array.isArray(events) ? events : [events],
Filter: renderFilters(filters),
};
```
### Other
I tried to use `bucket.addEventNotification(eventType, destination)` twice, but received the following error:
```typescript
myBucket.addEventNotification(OBJECT_CREATED, new s3n.SqsDestination(myQueue));
myBucket.addEventNotification(OBJECT_REMOVED, new s3n.SqsDestination(myQueue));
```
```
21/23 | 2:36:26 PM | CREATE_FAILED | Custom::S3BucketNotifications | stack/myBucket/Notifications (myBucketNotificationsABC975B5) Failed to create resource. Unable to validate the following destination configurations
More information in CloudWatch Log Stream: 2020/09/18/[$LATEST]41809b803bc2420e8d29cdde9641c471
```
From BucketNotification Custom Resource Lambda Function CloudWatch Logs, the `ResourceProperties` field (e.g. what gets translated into the PutBucketNotificationConfiguration request parameters) looks like this:
```json
"ResourceProperties": {
"ServiceToken": "arn:aws:lambda:us-west-2:123456789012:function:stack-BucketNotificationsHandler0-1PHGQG0C03BBS",
"BucketName": "stack-my-bucket465d0d98-1ciu90funjggu",
"NotificationConfiguration": {
"QueueConfigurations": [
{
"Events": [
"s3:ObjectCreated:*"
],
"QueueArn": "arn:aws:sqs:us-west-2:123456789012:my-queue"
},
{
"Events": [
"s3:ObjectRemoved:*"
],
"QueueArn": "arn:aws:sqs:us-west-2:123456789012:my-queue"
}
]
}
}
```
The problem is there are _two_ QueueConfigurations for _one_ SQS Queue (this also applies to SNS and Lambda destinations)
It "should" look like this:
```json
"QueueConfigurations": [
{
"Events": [
"s3:ObjectCreated:*",
"s3:ObjectRemoved:*"
],
"QueueArn": "arn:aws:sqs:us-west-2:123456789012:my-queue"
},
]
```
Reference Documentation:
* S3 API [QueueConfiguration#Events](https://docs.aws.amazon.com/AmazonS3/latest/API/API_QueueConfiguration.html#AmazonS3-Type-QueueConfiguration-Events)
* AWS JS SDK [S3.putBucketNotificationConfiguration](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putBucketNotificationConfiguration-property)
- [✅] :wave: I may be able to implement this feature request
... I mean it... this is super easy 😆
- [✅] :warning: This feature might incur a breaking change
---
This is a :rocket: Feature Request
Contributor guide
Research direction
Start with addEventNotification() in packages/@aws-cdk/aws-s3/lib/bucket.ts and addNotification() in packages/@aws-cdk/aws-s3/lib/notifications-resource/notifications-resource.ts. Inspect the existing S3 notification configuration tests or examples, then verify that both a single EventType and an EventType[] produce the expected configuration for one destination while preserving existing behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, typescript
- Domain
- cloud, infrastructure
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 50/100