Gracefully handle exceptions when waiting for Dataflow job completion.
Open
core
failing test
improvement
P3
python
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
If there is an error when trying to contact the dataflow service in Python's Dataflow.poll_for_job_completion, we may exit the thread prematurely.
A typical manifestation is: Dataflow Runner fails with:
```
AssertionError: Job did not reach to a terminal state after waiting indefinitely.
```
however job execution continues, and succeeds.
Imported from Jira [BEAM-6202](https://issues.apache.org/jira/browse/BEAM-6202). Original Jira may contain additional context.
Reported by: robertwb.
Contributor guide
Assessment
This issue has not been assessed yet.