googleapis / googleapis/google-cloud-ruby

BigQuery: Retry on 'internalError' when querying the results of a job

Open
#17,996 3 comments 0 reactions 0 assignees View on GitHub
api: bigquery type: feature request
Dominant language
Ruby
Stars
1.4k
Forks
578
Avg merge
1d 11h
Merged PRs (30d)
166

Description

**Is your feature request related to a problem? Please describe.**
Execution of BigQuery by `Google::Cloud::Bigquery::Dataset.query` sometimes fails due to `internalError` during waiting for completion.
To handle it, I need to implement retry besides an internal mechanism.

Precisely, `service.job_query_results` below sometimes raises ServerError saying "An internal error occurred" and "Retrying the job with back-off as described in the BigQuery SLA should solve the problem"
https://github.com/googleapis/google-cloud-ruby/blob/dce568ad26944f59d7d8a130c210a83abf410b53/google-cloud-bigquery/lib/google/cloud/bigquery/query_job.rb#L703

**Describe the solution you'd like**
Retry `internalError` inside the gem.
According to https://github.com/googleapis/google-cloud-ruby/issues/7846, internalError is not always retryable.
However, getting the result of a running job is not the case.
BigQuery's client says "An error occurred on the server and the request can be retried" too at https://github.com/googleapis/google-api-ruby-client/blob/c995f1627e9922a7a93a3f7f725ae596522fae69/google-api-client/generated/google/apis/bigquery_v2/service.rb#L418.

So, I would like this gem to retry execution when internalError happens while waiting for job completion.

**Describe alternatives you've considered**
It's possible to implement retry outside the gem.
But because it already has the mechanism, I believe retrying in one place is better.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.