confluentinc / confluentinc/parallel-consumer
Always process the freshest record when ordered by key
- Dominant language
- Java
- Stars
- 299
- Forks
- 172
- PR merge metrics
- No merged PRs in 30d
Description
Hello, I have a use case where I need to process each key sequentially however when processing a record, if new records arrive for that key, I would like to skip over the ones that have already arrives and process the latest one next.
I tried to read and understand the documentation but I believe currently the library does not support this kind of behavior. Am I wrong? If not, is there a not so hacky way to achieve this?
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reviewing the documentation and the key-sequential processing and queueing behavior described there. Compare the current behavior with the requested handling of newly arrived records, especially skipping older records for a key. Done means establishing whether the behavior is supported and documenting a supported approach or defining the implementation scope.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- backend, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100