apache / apache/beam

Data-driven triggers

Open
#17,981 0 comments 4 reactions 0 assignees View on GitHub
beam-model new feature P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

For some applications, it's useful to declare a pane/window to be emitted (or finished) based on its contents. The simplest of these is the AfterCount trigger, but more sophisticated predicates could be constructed.

The requirements for consistent trigger firing are essentially that the state of the trigger form a lattice and that the "should fire?" question is a monotonic predicate on the lattice. Basically it asks "are we high enough up the lattice?"

Because the element types may change between the application of Windowing and the actuation of the trigger, one idea is to extract the relevant data from the element at Windowing and pass it along implicitly where it can be combined and inspected in a type safe way later (similar to how timestamps and windows are implicitly passed with elements).

Imported from Jira [BEAM-101](https://issues.apache.org/jira/browse/BEAM-101). Original Jira may contain additional context.
Reported by: robertwb.

Contributor guide

Open the contributing guide

Research direction

Start by locating Apache Beam's Windowing and trigger APIs and the existing AfterCount trigger. Trace how timestamps and windows are passed with elements, then identify the tests and API design needed for data-driven predicates, type-safe propagation, and consistent trigger firing.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems, stream-processing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.