confluentinc / confluentinc/parallel-consumer

Always process the freshest record when ordered by key

Open
#902 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
299
Forks
172
PR merge metrics
No merged PRs in 30d

Description

Hello, I have a use case where I need to process each key sequentially however when processing a record, if new records arrive for that key, I would like to skip over the ones that have already arrives and process the latest one next.

I tried to read and understand the documentation but I believe currently the library does not support this kind of behavior. Am I wrong? If not, is there a not so hacky way to achieve this?

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the documentation and the key-sequential processing and queueing behavior described there. Compare the current behavior with the requested handling of newly arrived records, especially skipping older records for a key. Done means establishing whether the behavior is supported and documenting a supported approach or defining the implementation scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
java, kafka
Domain
backend, distributed-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.