apache / apache/beam

[Bug][Go SDK]: Dataflow status polling too optimistic

Open
#23,452 0 comments 0 reactions 0 assignees View on GitHub
bug go P3
Dominant language
Java
Stars
8.7k
Forks
4.7k
Avg merge
1d 20h
Merged PRs (30d)
196

Description

### What happened?

Polling for pipeline results from Dataflow can spuriously cause the local task to be marked as failed if the RPC itself fails.

```
2022/09/30 11:05:26 Job still running ...
ptest.go:108: Failed to execute job: failed to get job
caused by:
googleapi: Error 503: Deadline exceeded, backendError
```

Unless there's an auth error, the code should simply retry with backoff a few times, and at least annotate the error with that it's not a real pipeline job failure.

https://github.com/apache/beam/blob/master/sdks/go/pkg/beam/runners/dataflow/dataflowlib/job.go#L246

### Issue Priority

Priority: 3

### Issue Component

Component: sdk-go

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.