googleapis / googleapis/google-cloud-go
bigquery: add more network failure error response in Do function
- Dominant language
- Go
- Stars
- 4.5k
- Forks
- 1.6k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 109
Description
Some cases like poor network or long latency,
We encountered orphaned bq_load job when submit a job through HTTP POST and
receive socker recv failure while waiting response.
In this situation, Do function in bigquery-gen.go returns nil with recv failure error message but JOB
submitted and running. This brings duplicate job like 409 while running runWithRetryExplicit in insertJob function
1. The problem is error returned but JOB is running and not able to retrieve JOBID which is created by createJobRef function in job.go
to track status whether it is running, not or should retry on app layer.
2. This problem also brings duplicate job like 409 error while running runWithRetryExplicit in insertJob function. I'm not sure this brings duplicate data load but combination of case 1 and case 2, there are high possibilities and some customer encountered rarely.
**Describe the solution you'd like**
1. return error message using structured error code with message (ie: currently when library runs on windows and fails, it returns like: "wsarecv fail" but on linux it can "recv fail"). {"errorcode": -1, "msg": "wasrecv fail"} might useful
2. Let application do retry using their preferred way. Remove retry things inside sdk which is hard to track who sends duplicate job request.
Contributor guide
Assessment
This issue has not been assessed yet.