Range join in Beam
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
1.We can support some data-driven trigger, so we need expose data in OnElementContext of onElement method.
2.We can support more flexible join, so we need expose buffer tag in TriggerContext, now this buffer tag is in SystemReduceFn.
for example: SELECT STREAM * FROM Orders AS o JOIN Shipments AS s
ON o.orderId = s.orderId AND s.rowtime BETWEEN o.rowtime AND o.rowtime **** INTERVAL '1' HOUR;
link: https://issues.apache.org/jira/browse/BEAM-101
Imported from Jira [BEAM-912](https://issues.apache.org/jira/browse/BEAM-912). Original Jira may contain additional context.
Reported by: lzljs3620320.
Contributor guide
Research direction
Start with the linked Jira issues BEAM-101 and BEAM-912 for the missing context and range-join requirements. Then inspect OnElementContext, TriggerContext, and SystemReduceFn, which the issue identifies as the relevant entry points. Done means the required data and buffer tag are exposed so the described range join and data-driven trigger use cases can be supported.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, sql
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100