Transition to RUNNING only after offset-reset happens
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 373
- Forks
- 61
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
- Currently, ProcessorSubscription's state transitions to RUNNING at the end of rebalance listener (onPartitionsAssigned)
- So, offset-reset could be happen in later `poll()` after transitioned to RUNNING
- Sometimes, this could cause unintuitive behavior to the user.
* Even for us. We were not aware about this behavior until we found the benchmark was not running (https://github.com/line/decaton/pull/81)
### Example
- Some users set `auto.offset.reset = none` when they migrate their consumer to Decaton from other framework (which likely includes a lot of code changes), to make sure `group.id` is configured correctly so can continue consumption from committed offset.
* So they tried to detect the `group.id`-mistake by checking the subscription transitioned to RUNNING or not. (`auto.offset.reset = none` throws an exception when starts up if group.id doesn't exist, so they assumed the subscription doesn't transition to RUNNING if group.id is not configured properly)
- But, in fact, subscription transitions to RUNNING even if there's a mistake in group.id. (and transition to `SHUTTING_DOWN -> TERMINATED` soon)
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing ProcessorSubscription's state changes through the rebalance listener, especially onPartitionsAssigned, and the later poll() that performs offset reset. Reproduce the auto.offset.reset=none case with an incorrect group.id. Done means the subscription does not report RUNNING before offset reset completes, including when startup fails.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100