Merging of watermark holds depends on runner-specific "take the minimum" behavior
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
When multiple watermark holds are to be combined via an OutputTimeFn and committed to persistent storage, the case where the OutputTimeFn computes the minimum is elided, assuming the underlying runner will perform this combine by default.
This is a natural default, but today it is a runner-specific behavior that the runner-agnostic code relies upon.
Imported from Jira [BEAM-24](https://issues.apache.org/jira/browse/BEAM-24). Original Jira may contain additional context.
Reported by: kenn.
Contributor guide
Research direction
Start by locating the runner-agnostic code that combines watermark holds through an OutputTimeFn and persists the result. Compare the minimum case with other combine behaviors and identify where runner-specific behavior is assumed. Done means the minimum case is handled without relying on a runner-specific default, with relevant tests covering the merge.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems, stream-processing
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100