Revisit GroupByKey.createWithFewKeys
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 5h
- Merged PRs (30d)
- 204
Description
This doesn't have a parallel within the GroupByKeyPayload, so there's currently no way to send it through the Runner API.
The place it will almost always be created is in a `Combine.globally()`.
It's potentially useful as an optimizer hint. The Dataflow Runner in streaming mode disables combiner lifting unless the GroupByKey has the fewKeys property set to true.
Imported from Jira [BEAM-2620](https://issues.apache.org/jira/browse/BEAM-2620). Original Jira may contain additional context.
Reported by: tgroh.
Contributor guide
Research direction
Start by tracing GroupByKey.createWithFewKeys and the GroupByKeyPayload representation, then inspect how Combine.globally() constructs the transform and how the Dataflow Runner consumes the fewKeys hint. Done means the setting can be represented through the Runner API and remains usable as an optimizer hint for streaming combiner lifting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data, distributed-systems
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100