apache / apache/beam

TestPipeline.run should perhaps waitUntilFinish

Open
#18,352 0 comments 0 reactions 0 assignees View on GitHub
bug core flink java P3 runners tests
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
2d 2h
Merged PRs (30d)
205

Description

https://github.com/apache/beam/blob/master/sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipeline.java#L124

it calls waitUntilFinish() only if
1) run wasn't called
2) enableAutoRunIfMissing is true.

However in practice both of these are false.
1) run() is, in most tests, called. So effectively if you call .run() at all, then this thing doesn't call waitUntilFinish().
2) enableAutoRunIfMissing() is set to true only via TestPipeline.enableAutoRunIfMissing(), which is called only from its own unit test.

This means that, for all tests that use TestPipeline - if the test waits until finish, it's only because of the grace of the particular runner. Which is like really bad.

We're lucky because in practice TestDataflowRunner, TestApexRunner, TestSparkRunner in run() call themselves waitUntilFinish().

However, TestFlinkRunner doesn't - i.e. there currently might be tests that actually fail in Flink runner, undetected.

The proper fix to this is to fix TestPipeline to always waitUntilFinish().
Currently testing a quick-fix in https://github.com/apache/beam/pull/2240 to make sure Flink is safe.

Imported from Jira [BEAM-1712](https://issues.apache.org/jira/browse/BEAM-1712). Original Jira may contain additional context.
Reported by: jkff.

Contributor guide

Open the contributing guide

Research direction

Start with sdks/java/core/src/main/java/org/apache/beam/sdk/testing/TestPipeline.java at line 124 and inspect how run() and waitUntilFinish() interact. Compare the behavior with TestFlinkRunner and the other runners mentioned in the issue; done means TestPipeline always waits until finish after a pipeline is run, with Flink tests no longer able to complete early.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
testing
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.