confluentinc / confluentinc/parallel-consumer
feature: Option for batch to only contain messages of the same key
- Dominant language
- Java
- Stars
- 299
- Forks
- 172
- PR merge metrics
- No merged PRs in 30d
Description
Currently the batches are just a subset of non batched processing.
I.e.
- key based ordering - all keys in all batches will be unique.
- Unordered - no restrictions
Imagine a scenario where you have a topic where key is the account.
- You have a database per account.
- You want to do batch inserts to the database, where you do a batch of inserts (one insert per message) from within the batch.
- You don't want any messages from a different key, in this batch. All messages in this batch will be in a single external system call.
- Ordered inserting in db is still required. Key ordering is used. No other batch in flight is allowed to contain keys already in an inflight batch.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.