dbt-labs / dbt-labs/dbt-adapters

[Bug] Enforced contracts break recursive CTE rules

Open
#582 3 comments 1 reaction 0 assignees View on GitHub
feature:model-contracts help-wanted pkg:dbt-bigquery type:enhancement
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug in dbt-bigquery?

- [X] I believe this is a new bug in dbt-bigquery
- [X] I have searched the existing issues, and I could not find an existing issue for this bug

### Current Behavior

Recursive CTEs work in a model because can exist at the top-level. When enforcing contracts, a `select * from ()` is wrapped around the model query which breaks usage rules of BQ recursive CTEs.

### Expected Behavior

Models with recursive CTEs are able to build with enforced contracts

### Steps To Reproduce

1. Go to dbt Cloud > develop
2. Create a model called `test_model_v1.sql`
3. Paste this code into the model:
```sql
{{
config(
materialized='table'
)
}}

WITH RECURSIVE
CTE_1 AS (
(SELECT 1 AS iteration UNION ALL SELECT 1 AS iteration)
UNION ALL
SELECT iteration + 1 AS iteration FROM CTE_1 WHERE iteration < 3
)
SELECT iteration FROM CTE_1
ORDER BY 1 ASC
```
4. Create a _models.yml file and add configurations for the `test_model`:
```yaml
models:
- name: test_model
latest_version: 1
# config:
# contract:
# enforced: true
columns:
- name: iteration
data_type: int
tests:
- unique
- not_null
versions:
- v: 1
```
5. Run the model: `dbt run -s test_model`
6. Inspect the output:
![Screenshot 2024-02-06 at 1 54 36 PM](https://github.com/dbt-labs/dbt-adapters/assets/60892553/84552105-4741-444f-862e-ddd1ff597b67)
7. Go back to the `_models.yml` and uncomment the config for contracts
8. Rerun the model and inspect the output:
![Screenshot 2024-02-06 at 1 55 57 PM](https://github.com/dbt-labs/dbt-adapters/assets/60892553/4879c557-2446-4be4-94b1-911e9edbcbfb)

### Relevant log output

```shell
19:55:39 Began executing node model.dbt_bigquery_demo.test_model.v1
19:55:39 Opening a new connection, currently in state closed
19:55:39 On model.dbt_bigquery_demo.test_model.v1: select * from (

WITH RECURSIVE
CTE_1 AS (
(SELECT 1 AS iteration UNION ALL SELECT 1 AS iteration)
UNION ALL
SELECT iteration + 1 AS iteration FROM CTE_1 WHERE iteration < 3
)
SELECT iteration FROM CTE_1
ORDER BY 1 ASC
) as __dbt_sbq
where false and current_timestamp() = current_timestamp()
limit 0
19:55:39 BigQuery adapter: https://console.cloud.google.com/bigquery?project=sales-demo-project-314714&j=bq:US:106ef029-371d-4337-80a3-80b4650e88de&page=queryresults
19:55:39 BigQuery adapter: Retry attempt 1 of 1 after error: BadRequest('WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.; reason: invalidQuery, location: query, message: WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.')
19:55:40 BigQuery adapter: https://console.cloud.google.com/bigquery?project=sales-demo-project-314714&j=bq:US:9efd3b4e-62f1-48c1-bd00-0131d23f6321&page=queryresults
19:55:40 BigQuery adapter: https://console.cloud.google.com/bigquery?project=sales-demo-project-314714&j=bq:US:9efd3b4e-62f1-48c1-bd00-0131d23f6321&page=queryresults
19:55:40 Timing info for model.dbt_bigquery_demo.test_model.v1 (execute): 19:55:39.340818 => 19:55:40.648146
19:55:40 Database Error in model test_model (models/test_model_v1.sql)
WITH RECURSIVE is only allowed at the top level of the SELECT, CREATE TABLE AS SELECT, CREATE VIEW, INSERT, EXPORT DATA statements.
19:55:40 1 of 1 ERROR creating sql table model dbt_cberger_bigquery.test_model_v1 ....... [ERROR in 1.37s]
19:55:40 Finished running node model.dbt_bigquery_demo.test_model.v1
```

### Environment

```markdown
- OS: Mac
- Python:
- dbt-core: 1.7.7
- dbt-bigquery: 1.7.4
```

### Additional Context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.