dbt-labs / dbt-labs/dbt-adapters

[Bug] 🐞 dbt-bigquery does not retry on 503 error

Open
#941 5 comments 1 reaction 0 assignees View on GitHub
pkg:dbt-bigquery type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Summary

We intermittently get 503 errors from GCP/BigQuery, which result in job failures. These 503 do not get logged as [backend errors](https://cloud.google.com/bigquery/docs/error-messages). On the contrary, I see the job successfully finished in 0 seconds. The following shows up in "Bigquery Job explorer" section. I believe that is because this is **Runtime Error** as opposed to **Backend Error**.

```
Duration: 0 sec
Bytes processed: 0 B
Bytes billed: 0 B
Slot milliseconds: 0
```

### Logs:
```
1 of 4 ERROR creating sql view model myproject.my_dataset.my_table [ERROR in 1.36s]

BigQuery adapter: Unhandled error
Runtime Error in model my_model (dbt/models/some.sql)
503 GET https://bigquery.googleapis.com/bigquery/v2/projects/my-project/jobs/ffffffff-8d8d-4747-acac-eeeeeeeeeeee?location=US&prettyPrint=false:
Visibility check was unavailable. Please retry the request and contact support if the problem persists
```

### DBT Environment
```
dbt-core: 1.9.1
dbt-bigquery: 1.9.1
OS: debian docker image
Python: v3.10
```

### Expected Behavior

dbt retries on 503 when job_retries is set to 1

### Actual Behavior

dbt errors out without retrying even when job_retries is set to 1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.