[Feature Request]: [Go SDK] Encode Keys once for combiner lifting.
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What would you like to happen?
The Go SDK's combiner lifting currently encodes keys twice for caching. First it encodes values into a hash function, and then it repeats it to get the []byte for byte by byte comparisons in the cache.
https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/combine.go#L469
https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/combine.go#L483
For expensive keys, this can become very expensive without additional caching schemes.
Ideally this encoding happens once, instead of twice for each key being looked up.
Note: Windowed byte equality is important to use for final lookups because that's the only way to ensure equivalence to runners. Beam Model GBKs use byte equality for keys as their basis.
### Issue Priority
Priority: 2
### Issue Component
Component: sdk-go
Contributor guide
Assessment
This issue has not been assessed yet.