apache / apache/beam

[Feature Request][Go SDK] Support Key Aware CombineFns

Open
#22,211 0 comments 0 reactions 0 assignees View on GitHub
go new feature P2
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What would you like to happen?

It looks like that the CombineFns [documentation indicates that they should support being key aware.](https://pkg.go.dev/github.com/apache/beam/sdks/v2/go/pkg/beam#CombinePerKey)

However, validation doesn't support this style of CombineFn despite the execution code being present to handle it.

https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/core/runtime/exec/combine.go#L39

That it broke and there are no examples or tests, means it's never truly been supported, so this should be treated as a new feature.

Either the documentation and associated code should be cleared up to avoid future misunderstandings, or the feature should be properly implemented and integration tested.

Note that this would apply to all methods of a combineFn, as they may need to have different implementations depending on the key.

MergeAccumulators
AddInput
ExtractOutput
CreateAccumulator

### Issue Priority

Priority: 2

### Issue Component

Component: sdk-go

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.