Go SDK-Tune in memory pre-combine caching for Lifted Combines.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Requires [BEAM-4276](https://issues.apache.org/jira/browse/BEAM-4276) to be completed first.
Additional performance tweaks to the in memory per-key accumulator cache during the pre-combine phase of a lifted combine.
This can include any of
* capping the number of key-accumulator in the cache, and draining them eagerly after x elements seen, or evicting keys by some heuristic after the cache has seen X distinct keys.
* providing a counter on cache size (key/element counts) exposable through the metrics interface or another runner standard counter, to permit observing the cache's status, especially if it could grow without bound within a bundle.
Imported from Jira [BEAM-4468](https://issues.apache.org/jira/browse/BEAM-4468). Original Jira may contain additional context.
Reported by: lostluck.
Contributor guide
Assessment
This issue has not been assessed yet.