State API should allow state to be managed in different windows
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
For example, even if the elements are being processed in fixed windows of an hour, it may be desirable for the state to "roll over" between windows (or be available to all windows).
It will also be necessary to figure out when this state should be deleted (TTL? maximum retention?)
Another problem is how to deal with out of order data. If data comes in from the 10:00 AM window, should its state changes be visible to the data in the 9:00 AM window?
Imported from Jira [BEAM-1261](https://issues.apache.org/jira/browse/BEAM-1261). Original Jira may contain additional context.
Reported by: bchambers.
Contributor guide
Research direction
The issue names the State API and the original Jira ticket BEAM-1261, but no files, tests, or entry points. Start by reviewing the State API and the linked Jira context; done requires an agreed design for cross-window visibility, retention or deletion, and out-of-order updates.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100