apache / apache/beam

[Task][Go SDK]: Add github precommits mini suites for each runner against the Go SDK

Open
#27,748 1 comment 0 reactions 0 assignees View on GitHub
github_actions go infra P3 stale task
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What needs to happen?

The Go SDK suites are increasingly comprehensive, and execution against the default runner, and the Python Portable runner tends to catch most issues that would break in post commit.

The improvement would be to have a selected suite of a handful of pipelines that would then run in each individual runner as precommits. The validates runner suites are too heavyweight to run against every runner on every PR, but reducing to a small set of tests would be a good smoke check against other changes for the testing suites and so on.

Eg. In particular, the dataflow path in the validates_runner script has additional logic compared to the other, so even a small precommit check there would avoid accidentally breaking the postcommit.

The validates runner script already has a --tests flag (defaulting to `"./test/integration/... ./test/regression"`) which could be used when invoking the script from a precommit, to reduce the suite to a select instance. Or even the --pipeline_opts flag could be used to add a -run= addition to the `go test` invocation.

### Issue Priority

Priority: 3 (nice-to-have improvement)

### Issue Components

- [ ] Component: Python SDK
- [ ] Component: Java SDK
- [X] Component: Go SDK
- [ ] Component: Typescript SDK
- [ ] Component: IO connector
- [ ] Component: Beam examples
- [ ] Component: Beam playground
- [ ] Component: Beam katas
- [ ] Component: Website
- [ ] Component: Spark Runner
- [ ] Component: Flink Runner
- [ ] Component: Samza Runner
- [ ] Component: Twister2 Runner
- [ ] Component: Hazelcast Jet Runner
- [ ] Component: Google Cloud Dataflow Runner

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.