[Bug][Go SDK]: Dataflow status polling too optimistic
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 196
Description
### What happened?
Polling for pipeline results from Dataflow can spuriously cause the local task to be marked as failed if the RPC itself fails.
```
2022/09/30 11:05:26 Job still running ...
ptest.go:108: Failed to execute job: failed to get job
caused by:
googleapi: Error 503: Deadline exceeded, backendError
```
Unless there's an auth error, the code should simply retry with backoff a few times, and at least annotate the error with that it's not a real pipeline job failure.
https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go#L246
### Issue Priority
Priority: 3
### Issue Component
Component: sdk-go
Contributor guide
Assessment
This issue has not been assessed yet.