dbt-labs / dbt-labs/dbt-adapters

[Bug] Can't use data contract constraints together with 'time_ingestion_partitioning="true"' in incremental models

Open
#525 4 comments 2 reactions 0 assignees View on GitHub
pkg:dbt-bigquery triage:product type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-bigquery?

- [X] I believe this is a new bug in dbt-bigquery
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

Dbt errors when combining an incremental model with `time_ingestion_partitioning` enabled and also enforcing model contract constraints.

I'd love to know whether this is intended or if this is potentially a bug that could be worked on :)

### Expected Behavior

Dbt should not error out on columns 'consumed' by the `time_ingestion_partitioning` flags that are in the dbt model but not factually in the output schema or BigQuery table when enforcing [model contract constraints](https://docs.getdbt.com/reference/resource-properties/constraints#platform-specific-support).

The partitioning column should not have 'presence enforcement' when this flag is enabled (ideally).

### Steps To Reproduce

```
# incremental_test.sql
{{ config(
materialized="incremental",
on_schema_change="fail",
partition_by={
"field": "created_date",
"data_type": "timestamp",
"granularity": "day",
"time_ingestion_partitioning": true,
}
) }}

SELECT
'A' AS letter,
1 AS digit,
CURRENT_DATE() AS created_date
```
When enforcing the data contract like so:

```
# incremental_test.yaml
models:
- name: incremental_test
config:
contract:
enforced: true
columns:
- name: letter
data_type: STRING
constraints:
- type: not_null
- name: digit
data_type: INTEGER
constraints:
- type: not_null
```

and subsequently building with `dbt build --select incremental_test` I get the error attached in the log output.

### Relevant log output

```shell
This model has an enforced contract that failed.
Please ensure the name, data_type, and number of columns in your contract match the columns in your model's definition.

| column_name | definition_type | contract_type | mismatch_reason |
| ------------ | --------------- | ------------- | ------------------- |
| created_date | DATE | | missing in contract |
```
```

### Environment

```markdown
- OS: dbt-cloud (through dbt Cloud CLI 0.38.18)
- Python: 3.12
- dbt-core: 1.9 (latest)
- dbt-bigquery: 1.9.0
```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.