apache / apache/beam

[GoSDK] There should be a combineFn unit tester that validates lifting.

Open
#19,262 0 comments 0 reactions 0 assignees View on GitHub
go improvement P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

The Go direct runner doesn't handle combiner lifting, which means combiners aren't validated whether they support lifting correctly or not. It should be possible for the ptest package to simulate combiner lifting, and actuate the lifecycle methods, or for the direct runner to be re-written to handle combiner lifting as a configuration option. This would make it easier to avoid mistakes and typos when handling lifecycle methods for CombineFns.

 

In particular look at [https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/transforms/top/top_test.go](https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/transforms/top/top_test.go) which could be much simpler for users implementing combiners and avoid the manual scaffolding.

Imported from Jira [BEAM-6682](https://issues.apache.org/jira/browse/BEAM-6682). Original Jira may contain additional context.
Reported by: lostluck.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.