awslabs / awslabs/amazon-sqs-java-extended-client-lib

Retention of S3 object once message consumed by queue.

Open
#40 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
232
Forks
113
PR merge metrics
No merged PRs in 30d

Description

One part of the outstanding pull request https://github.com/awslabs/amazon-sqs-java-extended-client-lib/pull/6 addresses this.

If a queue is one of many subscribers to a topic, then we want all the queues to have a 'fair go' at processing a message before the max retention period of the topic lapses.

With the current setup, the deletion of a message from the queue using the extended lib deletes the underlying S3 bucket. This is fine when the queue is the only consumer of the message (and implicitly the S3 bucket) but with a topic and queue setup the first consumer will stop any other subscriber from getting the contents of the S3 bucket.

We can currently work around this by writing custom code to override the behaviour of the client when deleting a message, but this seems flaky given we don't control the library itself.
Would be great to get a configuration option that leaves the message in S3 when a consumer sends a delete message request . This way we could let the topic drive when an S3 bucket is finally cleaned up.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.