dbt-labs / dbt-labs/dbt

[v2 Bug] BigQuery: `priority: batch` profile configuration is ignored (all queries submitted as INTERACTIVE)

Open
#16,021 1 comment 0 reactions 1 assignee Claimed by @serramatutu View on GitHub
adapter:bigquery area:adapters
Dominant language
Rust
Stars
13.8k
Forks
2.6k
Avg merge
21h 31m
Merged PRs (30d)
56

Description

### Is this a new bug in dbt v2.x compared to the latest version of dbt 1.x?
- [x] I believe this is a new bug in dbt v2.x
- [x] I have searched the existing issues and could not find a duplicate

### Current Behavior
When configuring a BigQuery target with `priority: batch` in `profiles.yml`, `dbt debug` parses the setting and displays `"priority": "batch"`, but all queries submitted to BigQuery are executed with `priority: "INTERACTIVE"` (verified via BigQuery `INFORMATION_SCHEMA.JOBS_BY_PROJECT`).

### Expected Behavior
When `priority: batch` is set on a BigQuery profile target, dbt Fusion should configure query jobs with `priority: "BATCH"` (mirroring dbt Core 1.x behavior), allowing queries to run via BigQuery's batch queue.

### Steps To Reproduce
1. Configure `profiles.yml` with a BigQuery target using `priority: batch`:
```yaml
my_profile:
target: dev
outputs:
dev:
type: bigquery
method: oauth
project: my-project
dataset: my_dataset
priority: batch
```
2. Run `dbt run --select ` using dbt Fusion (`2.0.0-preview.203` or `2.0.0-preview.210` / `2.0.0-beta.2`).
3. Check the resulting job priority in BigQuery:
```sql
SELECT job_id, priority, query
FROM `region-us-central1`.`INFORMATION_SCHEMA.JOBS_BY_PROJECT`
WHERE creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 5 MINUTE)
ORDER BY creation_time DESC;
```
4. Notice that `priority` is `INTERACTIVE` rather than `BATCH`.

In contrast, running the same model using Python dbt Core 1.x (`dbt-bigquery`) generates a job with `priority = "BATCH"`.

### Root Cause / Code Analysis
- In `crates/dbt-schemas/src/schemas/profiles.rs`, `BigqueryDbConfig` includes `pub priority: Option`.
- In `crates/dbt-adbc/src/bigquery.rs`, `pub const QUERY_PRIORITY: &str = "adbc.bigquery.sql.query.priority";` is defined for Arrow ADBC.
- However, `QUERY_PRIORITY` / `adbc.bigquery.sql.query.priority` is not set on the ADBC statement/connection options when executing SQL statements, so the underlying BigQuery ADBC driver defaults all jobs to `INTERACTIVE`.

### Environment
- OS: macOS / Linux
- CPU: ARM64 / x86_64
- dbt distribution and version: `dbt-fusion 2.0.0-preview.203` and `2.0.0-preview.210` (`v2.0.0-beta.2`)

### Which database adapter are you using?
bigquery

### Is this a discrepancy vs. dbt 1.x?
- [x] Yes — this works in dbt 1.x but not in dbt v2.x

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.