aws / aws/aws-cdk

[S3] Support Multiple S3 Object Notification Event Types for a single Event Notification Configuration

Open
#10,439 4 comments 8 reactions 0 assignees View on GitHub
@aws-cdk/aws-s3 bug effort/small p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
1d 19h
Merged PRs (30d)
74

Description

The `S3.PutBucketNotificationConfiguration` API supports an _Array of Strings_ for the `EventType` field, however currently the AWS CDK only allows passing a single [EventType enum](https://github.com/aws/aws-cdk/blob/f0c76ac1f930fcbe7a2610e7aeeb4a46721516e1/packages/@aws-cdk/aws-s3/lib/bucket.ts#L1687-L1790) value into the `@aws-cdk/aws-s3/bucket#addEventNotification()` function.

The `addEventNotification()` function should support an Array of S3 Event Types (`EventType[]`) to pass to the S3 PutBucketNotificationConfiguration API.

### Use Case

To support multiple event types for a single S3 Bucket Notification Configuration.

I need to have an SQS Queue or SNS Topic which receives both Object Created events as well as Object Deleted Events.

### Proposed Solution

Support an (optional?) array of events:

`packages/@aws-cdk/aws-s3/lib/notifications-resource/notifications-resource.ts#L51-L60`:

https://github.com/aws/aws-cdk/blob/90de605a0f9c5487c302e34db5f89dcd58e3b445/packages/@aws-cdk/aws-s3/lib/notifications-resource/notifications-resource.ts#L51-L60

`packages/@aws-cdk/aws-s3/lib/bucket.ts#L1319-L1321`:

https://github.com/aws/aws-cdk/blob/f0c76ac1f930fcbe7a2610e7aeeb4a46721516e1/packages/@aws-cdk/aws-s3/lib/bucket.ts#L1319-L1321

The signature from `notifications-resource.ts` could support backwards compatibility:

```typescript
public addNotification(events: EventType | EventType[], target: IBucketNotificationDestination, ...filters: NotificationKeyFilter[]) {

// ... snip ...

// If `events` isn't an Array, add the single event to an array to support backwards compatibility.
const commonConfig: CommonConfiguration = {
Events: Array.isArray(events) ? events : [events],
Filter: renderFilters(filters),
};
```

### Other

I tried to use `bucket.addEventNotification(eventType, destination)` twice, but received the following error:

```typescript
myBucket.addEventNotification(OBJECT_CREATED, new s3n.SqsDestination(myQueue));
myBucket.addEventNotification(OBJECT_REMOVED, new s3n.SqsDestination(myQueue));
```

```
21/23 | 2:36:26 PM | CREATE_FAILED | Custom::S3BucketNotifications | stack/myBucket/Notifications (myBucketNotificationsABC975B5) Failed to create resource. Unable to validate the following destination configurations
More information in CloudWatch Log Stream: 2020/09/18/[$LATEST]41809b803bc2420e8d29cdde9641c471
```

From BucketNotification Custom Resource Lambda Function CloudWatch Logs, the `ResourceProperties` field (e.g. what gets translated into the PutBucketNotificationConfiguration request parameters) looks like this:

```json
"ResourceProperties": {
"ServiceToken": "arn:aws:lambda:us-west-2:123456789012:function:stack-BucketNotificationsHandler0-1PHGQG0C03BBS",
"BucketName": "stack-my-bucket465d0d98-1ciu90funjggu",
"NotificationConfiguration": {
"QueueConfigurations": [
{
"Events": [
"s3:ObjectCreated:*"
],
"QueueArn": "arn:aws:sqs:us-west-2:123456789012:my-queue"
},
{
"Events": [
"s3:ObjectRemoved:*"
],
"QueueArn": "arn:aws:sqs:us-west-2:123456789012:my-queue"
}
]
}
}
```

The problem is there are _two_ QueueConfigurations for _one_ SQS Queue (this also applies to SNS and Lambda destinations)

It "should" look like this:

```json
"QueueConfigurations": [
{
"Events": [
"s3:ObjectCreated:*",
"s3:ObjectRemoved:*"
],
"QueueArn": "arn:aws:sqs:us-west-2:123456789012:my-queue"
},
]

```

Reference Documentation:

* S3 API [QueueConfiguration#Events](https://docs.aws.amazon.com/AmazonS3/latest/API/API_QueueConfiguration.html#AmazonS3-Type-QueueConfiguration-Events)
* AWS JS SDK [S3.putBucketNotificationConfiguration](https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#putBucketNotificationConfiguration-property)

- [✅] :wave: I may be able to implement this feature request
... I mean it... this is super easy 😆

- [✅] :warning: This feature might incur a breaking change

---

This is a :rocket: Feature Request

Contributor guide

Open the contributing guide

Research direction

Start with addEventNotification() in packages/@aws-cdk/aws-s3/lib/bucket.ts and addNotification() in packages/@aws-cdk/aws-s3/lib/notifications-resource/notifications-resource.ts. Inspect the existing S3 notification configuration tests or examples, then verify that both a single EventType and an EventType[] produce the expected configuration for one destination while preserving existing behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
cloud, infrastructure
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.