awslabs / awslabs/aws-solutions-constructs

S3ToStepFunctions: add a SQS FIFO Queue to get exactly-once delivery

Open
#794 3 comments 1 reaction 0 assignees View on GitHub
feature-request needs-triage
Dominant language
TypeScript
Stars
1.4k
Forks
268
Avg merge
5h 18m
Merged PRs (30d)
5

Description

In using this construct I found I was spending a lot more than expected because I was getting many duplicate events, and running my ML step function is quite expensive. I suggest adding a FIFO queue to get exactly-once delivery semantics (or change to an S3-SNS-SQS-Stepfunction workflow).

This is a :rocket: Feature Request

Contributor guide

Open the contributing guide

Research direction

Start at the S3ToStepFunctions construct and inspect its current event-delivery path. Compare the proposed SQS FIFO option with the alternative S3-SNS-SQS-StepFunctions workflow, then define which approach provides the requested delivery semantics and what behavior should demonstrate completion.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, typescript
Domain
backend, cloud
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.