Asynchronous producers in Kinesis
- Dominant language
- Ruby
- Stars
- 3.7k
- Forks
- 1.2k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 5
Description
### Issue description
Currently Kinesis API within the AWS Ruby SDK only supports synchronous producers. While benchmarking internally we found the latency of put to be around 8ms on average. This seems a lot and ideally an application using Kinesis a lot will this quite fast. So, I was looking for asynchronous producers on Kinesis, it doesn't seem like Ruby MRI has a library that supports that. From Kinesis docs it seems like the performant way to use that is KPL but that has interfaces available only for Java so that rules it out.
Do you guys have any suggestion on this? I was thinking of implementing a side extension to the main lib that comes with aws-sdk-kinesis where the main thread just pushes the job onto a list which has a worker thread which takes care of publishing messages to Kinesis asynchronously. The current approach is a fire and forget approach where you won't get acknowledgement of message deliveries and can lead to dropped messages in case like when Kinesis's APIs are down.
### Gem name
aws-sdk, aws-sdk-kinesis
### Version of Ruby
Ruby: 2.4/2.5
Contributor guide
Assessment
This issue has not been assessed yet.