Prevent consuming segment from entering ERROR state due to stream connection errors
- Dominant language
- Java
- Stars
- 6.1k
- Forks
- 1.5k
- Avg merge
- 1d 21h
- Merged PRs (30d)
- 189
Description
## Background
In the event of consumption issues, a consuming segment sends a "stopped consumption" message to the controller, which then marks that consuming segment as OFFLINE in the Ideal State. This approach allows us to recover from transient stream errors through the RealtimeSegmentValidationManager periodic job. This job creates a new consuming segment in the Ideal State if the last segment of a stream partition is in the OFFLINE state.
## Problem Statement
The scenario described in the previous section comes into play after consumption has started, which is when the OFFLINE to CONSUMING transition is completed. During this transition, stream consumers are set up to collect some metadata from the stream. However, if there is any exception thrown while connecting to the stream, the segment goes into the ERROR state in the External View. Unfortunately, this doesn't align with the mentioned automatic recovery process using the RealtimeSegmentValidationManager job.
Contributor guide
Research direction
Start by tracing the OFFLINE to CONSUMING transition and the RealtimeSegmentValidationManager recovery job described in the issue. Inspect how stream connection exceptions affect the segment's External View; done means those errors follow the OFFLINE recovery path instead of leaving the segment in ERROR.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- databases, distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100