Optimize precombining table.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
The current code constructs, hashes, and compares full WindowedValues for the grouping key, which ends up dominating the time spent in the combining table when using trivial combiners (like Sum.integers()). We only need compare the (structural value of the) key and windows, and can emit the windows in the global case.
Imported from Jira [BEAM-14464](https://issues.apache.org/jira/browse/BEAM-14464). Original Jira may contain additional context.
Reported by: robertwb.
Contributor guide
Research direction
Start by locating the combining table and the construction, hashing, and comparison of WindowedValues. Compare behavior and performance with trivial combiners such as Sum.integers(), including the global case; done means grouping remains correct while avoiding unnecessary full-value work.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering, performance
- Issue type
- Refactor
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100