dbt-labs / dbt-labs/dbt-adapters
[Feature] update dbt-bigquery retry options
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this your first time submitting a feature request?
- [x] I have read the [expectations for open source contributors](https://docs.getdbt.com/docs/contributing/oss-expectations)
- [x] I have searched the existing issues, and I could not find an existing issue for this feature
- [x] I am requesting a straightforward extension of existing dbt functionality, rather than a Big Idea better suited to a discussion
### Describe the feature
Expose 2 new parameters for the bigquery retry functionality - give google some more time to spin up or find resources:
1. base_retry_delay
2. exponential_sleep_multiplier
https://cloud.google.com/storage/docs/retry-strategy
(I did not find a similar page specific to BQ)
Bigquery has issues with jobs and resources not being available.
About 1/1000 bq jobs randomly seems to fail. Retrying the job solves most of those issues.
But , sometimes google just needs some more time.
Configure a delay and stop the random job failures.
### Describe alternatives you've considered
Manually restarting the job an hour later normally solves everything.
### Who will this benefit?
Any user of dbt-bigquery with a large n of models.
"The Business". Stakeholders don't seem to care that google had an intermittent issue. SLA's are brutal.
### Are you interested in contributing this feature?
Sure!
### Anything else?
_No response_
Contributor guide
Assessment
This issue has not been assessed yet.