[v2 Bug] BigQuery: `priority: batch` profile configuration is ignored (all queries submitted as INTERACTIVE)
- Dominant language
- Rust
- Stars
- 13.8k
- Forks
- 2.6k
- Avg merge
- 21h 31m
- Merged PRs (30d)
- 56
Description
### Is this a new bug in dbt v2.x compared to the latest version of dbt 1.x?
- [x] I believe this is a new bug in dbt v2.x
- [x] I have searched the existing issues and could not find a duplicate
### Current Behavior
When configuring a BigQuery target with `priority: batch` in `profiles.yml`, `dbt debug` parses the setting and displays `"priority": "batch"`, but all queries submitted to BigQuery are executed with `priority: "INTERACTIVE"` (verified via BigQuery `INFORMATION_SCHEMA.JOBS_BY_PROJECT`).
### Expected Behavior
When `priority: batch` is set on a BigQuery profile target, dbt Fusion should configure query jobs with `priority: "BATCH"` (mirroring dbt Core 1.x behavior), allowing queries to run via BigQuery's batch queue.
### Steps To Reproduce
1. Configure `profiles.yml` with a BigQuery target using `priority: batch`:
```yaml
my_profile:
target: dev
outputs:
dev:
type: bigquery
method: oauth
project: my-project
dataset: my_dataset
priority: batch
```
2. Run `dbt run --select ` using dbt Fusion (`2.0.0-preview.203` or `2.0.0-preview.210` / `2.0.0-beta.2`).
3. Check the resulting job priority in BigQuery:
```sql
SELECT job_id, priority, query
FROM `region-us-central1`.`INFORMATION_SCHEMA.JOBS_BY_PROJECT`
WHERE creation_time >= TIMESTAMP_SUB(CURRENT_TIMESTAMP(), INTERVAL 5 MINUTE)
ORDER BY creation_time DESC;
```
4. Notice that `priority` is `INTERACTIVE` rather than `BATCH`.
In contrast, running the same model using Python dbt Core 1.x (`dbt-bigquery`) generates a job with `priority = "BATCH"`.
### Root Cause / Code Analysis
- In `crates/dbt-schemas/src/schemas/profiles.rs`, `BigqueryDbConfig` includes `pub priority: Option`.
- In `crates/dbt-adbc/src/bigquery.rs`, `pub const QUERY_PRIORITY: &str = "adbc.bigquery.sql.query.priority";` is defined for Arrow ADBC.
- However, `QUERY_PRIORITY` / `adbc.bigquery.sql.query.priority` is not set on the ADBC statement/connection options when executing SQL statements, so the underlying BigQuery ADBC driver defaults all jobs to `INTERACTIVE`.
### Environment
- OS: macOS / Linux
- CPU: ARM64 / x86_64
- dbt distribution and version: `dbt-fusion 2.0.0-preview.203` and `2.0.0-preview.210` (`v2.0.0-beta.2`)
### Which database adapter are you using?
bigquery
### Is this a discrepancy vs. dbt 1.x?
- [x] Yes — this works in dbt 1.x but not in dbt v2.x
Contributor guide
Assessment
This issue has not been assessed yet.