googleapis / googleapis/google-cloud-go

bigquery: add more network failure error response in Do function

Open
#8,767 0 comments 0 reactions 2 assignees Claimed by @alvarowolfx View on GitHub
api: bigquery priority: p2 type: feature request
Dominant language
Go
Stars
4.5k
Forks
1.6k
Avg merge
1d 13h
Merged PRs (30d)
109

Description

Some cases like poor network or long latency,
We encountered orphaned bq_load job when submit a job through HTTP POST and
receive socker recv failure while waiting response.
In this situation, Do function in bigquery-gen.go returns nil with recv failure error message but JOB
submitted and running. This brings duplicate job like 409 while running runWithRetryExplicit in insertJob function
1. The problem is error returned but JOB is running and not able to retrieve JOBID which is created by createJobRef function in job.go
to track status whether it is running, not or should retry on app layer.
2. This problem also brings duplicate job like 409 error while running runWithRetryExplicit in insertJob function. I'm not sure this brings duplicate data load but combination of case 1 and case 2, there are high possibilities and some customer encountered rarely.

**Describe the solution you'd like**
1. return error message using structured error code with message (ie: currently when library runs on windows and fails, it returns like: "wsarecv fail" but on linux it can "recv fail"). {"errorcode": -1, "msg": "wasrecv fail"} might useful
2. Let application do retry using their preferred way. Remove retry things inside sdk which is hard to track who sends duplicate job request.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.