Batching DoFn
Open
core
new feature
P3
python
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
A proper batching DoFn should remember the window associated with each element, hold the timestamp back, and re-emit the results with the windows/timestamps of the inputs
cc: [~robertwb]
Imported from Jira [BEAM-2548](https://issues.apache.org/jira/browse/BEAM-2548). Original Jira may contain additional context.
Reported by: altay.
Contributor guide
Research direction
Start with the imported BEAM-2548 Jira issue, then locate the DoFn batching entry point in Apache Beam. Trace how elements, windows, and timestamps are currently handled; done means each element's window is retained, its timestamp is held back, and emitted results restore the corresponding input windows and timestamps.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- stream-processing
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100