dbt-labs / dbt-labs/dbt-adapters

[Bug] BigQuery: Casting partition column disables partition pruning, causing expensive query execution

Open
#1,117 2 comments 1 reaction 0 assignees View on GitHub
triage:product type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-core?

- [x] I believe this is a new bug in dbt-core
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

**Description**:
When using dbt incremental or microbatching strategies on partitioned BigQuery tables, filters on the partition column (e.g. `timemsc`) are sometimes casted to timestamp. This disables partition pruning and results in very high query costs and execution time.

**Observed Behavior**:

* Query without cast (good): processed 137.82 MB
* Query with cast (bad): processed 736.15 GB

```sql
-- Good
where timemsc >= '2025-04-20' and timemsc < '2025-04-21'

-- Bad
where cast(timemsc as timestamp) >= '2025-04-20 00:00:00+00:00'
and cast(timemsc as timestamp) < '2025-04-21 00:00:00+00:00'
```

### Expected Behavior

The dbt framework should avoid applying casts to partition columns when filtering on date/datetime boundaries

### Steps To Reproduce

1. Use dbt with `incremental_strategy = 'insert_overwrite'` or `microbatching`
2. Model filters on a partitioned BigQuery table with a timestamp column
3. Observe query structure and bytes processed

### Relevant log output

```shell

```

### Environment

```markdown
* dbt version: Versionelless
* Adapter: dbt-bigquery
* Warehouse: BigQuery Standard SQL
```

### Which database adapter are you using with dbt?

_No response_

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.