aws / aws/aws-dotnet-messaging
Batch `DeleteMessage` to substantially increase throughput
- Dominant language
- C#
- Stars
- 143
- Forks
- 27
- Avg merge
- 1d 18h
- Merged PRs (30d)
- 4
Description
### Describe the feature
Batch `DeleteMessage` on successful handler invocations to reduce both cost and API load, raising FIFO processing limits 5.5 times.
### Use Case
Both `ReceiveMessage` and `DeleteMessage` count towards the 300 TPS on FIFO queues.
Under load, `RecevieMessage` returns 10 messages which are then processed by individual handler invocations. Each handler calls `DeleteMessage` on completion -a total of 11 API calls per 10 messages.
The TPS count with this workflow limits message throughput to ~270 messages per second on FIFO queues and drives up the API cost unnecessarily.
### Proposed Solution
Instead of each handler calling `DeleteMessage`, it would be preferrable to batch the receipts for a single API call of 10 messages or [n] milliseconds (whichever happens first).
At maximum rate; message processing on a FIFO queue would increase from ~272 messages per second (300 / 11 * 10) to ~1500 (300 / 2 * 10) before being throttled.
### Other Information
_No response_
### Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### AWS.Messaging (or related) package versions
AWS.Messaging.*
### Targeted .NET Platform
all supported
### Operating System and version
n/a
Contributor guide
Research direction
Start by reading the AWS.Messaging handler flow around ReceiveMessage and successful handler completion, then inspect how DeleteMessage is currently invoked. Identify the SQS batching entry points and determine how a receipt batch and count-or-time trigger would be represented; done means successful receipts are deleted in batches without changing handler semantics.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, csharp
- Domain
- cloud, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100