Implement real synchronized processing time in Flink runner
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
The "continuation trigger" for a processing time trigger is a synchronized processing time trigger. Today, this throws an exception in the FlinkRunner.
The supports the following:
- GBK1
- GBK2
When GBK1 fires due to processing time past the first element in the pane and that element arrives at GBK2, it will wait until all the other upstream keys have also processed and emitted corresponding data.
Sorry for the terseness of explanation - writing quickly so I don't forget.
Imported from Jira [BEAM-1641](https://issues.apache.org/jira/browse/BEAM-1641). Original Jira may contain additional context.
Reported by: kenn.
Contributor guide
Research direction
Start in the FlinkRunner at the exception raised for the continuation trigger, then trace the synchronized processing-time trigger behavior through GBK1 and GBK2. Done means processing-time firing after the first pane element waits for corresponding data from all upstream keys instead of throwing an exception.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, distributed-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100