Fail Dataflow Streaming pipeline on passert failure
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
Dataflow retries a failed work item indefinitely.
As PAsserts are translated to PTransforms, A failed PAssert will be retried indefinitely in Streaming mode.
So we should stop fail the pipeline in case of PAssert failure instead of waiting for test timeout.
Example job [https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-west1/jobs/2020-01-09_13_25_08-2233620052523810338?project=apache-beam-testing&folder&organizationId](https://pantheon.corp.google.com/dataflow/jobsDetail/locations/us-west1/jobs/2020-01-09_13_25_08-2233620052523810338?project=apache-beam-testing&folder&organizationId)
Imported from Jira [BEAM-9081](https://issues.apache.org/jira/browse/BEAM-9081). Original Jira may contain additional context.
Reported by: angoenka.
Contributor guide
Assessment
This issue has not been assessed yet.