googleapis / googleapis/google-cloud-python

Job created from library doesn't respect project default `default_query_job_timeout_ms` config

Open
#15,719 6 comments 0 reactions 1 assignee Claimed by @gkevinzheng View on GitHub
api: bigquery priority: p3 type: bug
Dominant language
Python
Stars
5.4k
Forks
1.8k
Avg merge
3d 4h
Merged PRs (30d)
122

Description

### Expected behaviour
BQ jobs created/initated via this library should respect the `default_query_job_timeout_ms` set on the BQ project.

### Issue
Hey team, just raising an inquiry about the second half of my comment raised here https://github.com/dbt-labs/dbt-bigquery/issues/231. From what I can tell, a query created from this library does not respect the `default_query_job_timeout_ms` set on the BQ project for some reason.

#### Environment details

- OS type and version: macOS / Ventura 13.4.1
- Python version: `Python 3.9.13`
- pip version: `pip 23.2.1`
- `google-cloud-bigquery` version:

```
name: google-cloud-bigquery
Version: 3.11.4
Summary: Google BigQuery API client library
Home-page: https://github.com/googleapis/python-bigquery
Author: Google LLC
Author-email: googleapis-packages@google.com
License: Apache 2.0
Location: /Users/jeremy/src/dbt-basic/venv_dbt_1.5.latest/lib/python3.9/site-packages
Requires: google-api-core, google-cloud-core, google-resumable-media, grpcio, packaging, proto-plus, protobuf, python-dateutil, requests
Required-by: dbt-bigquery
```

#### Steps to reproduce

1. Set the `default_query_job_timeout_ms` to be 5 minutes:

```sql
alter project `cse-sandbox-319708` set options (`region-us.default_query_job_timeout_ms` = 300000);
```

2. Run a query/job from the BQ UI that takes > 5 minutes:

![image](https://github.com/googleapis/google-cloud-python/assets/8496748/daa2a1e8-f29a-4baa-b90a-610729bbdd1e)

Notice that we run into an error at 5 minutes - as expected.

3. Run a query/job from python using this library that takes > 5 minutes (see Python code example below):

![image](https://github.com/googleapis/google-cloud-python/assets/8496748/d046dacc-0ca9-40fd-9458-3e0029257e24)

Notice that on the BQ side - the query/job completes taking > 5 minutes (6 m 41 s) - this is unexpected.

#### Code example

```python
from google.cloud import bigquery

client = bigquery.Client()

# Perform a query.
query = """
call dbt_jyeo.fake_sleep(400);
select * from `region-us.INFORMATION_SCHEMA.PROJECT_OPTIONS`;
"""

query_job = client.query(query) # API request
rows = query_job.result() # Waits for query to finish

for row in rows:
print(row)
```

#### Stack trace
There are no errors per se so nothing to call out here.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.