dbt-labs / dbt-labs/dbt-adapters

Ephemeral model injection causes syntax error when using WITH expression syntax (variable declaration)

Open
#1,422 0 comments 0 reactions 0 assignees View on GitHub
triage:product type:bug
Dominant language
Python
Stars
233
Forks
362
Avg merge
3d 22h
Merged PRs (30d)
9

Description

### Is this a new bug?

- [x] I believe this is a new bug
- [x] I have searched the existing issues, and I could not find an existing issue for this bug

### Which packages are affected?

- [x] dbt-adapters
- [ ] dbt-tests-adapter
- [ ] dbt-athena
- [ ] dbt-athena-community
- [x] dbt-bigquery
- [ ] dbt-postgres
- [ ] dbt-redshift
- [ ] dbt-snowflake
- [ ] dbt-spark

### Current Behavior

When an ephemeral model is referenced inside a model that uses BigQuery's WITH expression syntax (for local variable declarations, not CTEs), dbt incorrectly injects the ephemeral model definition inside the WITH expression block.

This leads to a compilation syntax error because the compiler confuses the WITH expression with a standard WITH CTE clause.

### Expected Behavior

The compiled SQL should correctly handle the WITH expression and inject the ephemeral model as a CTE outside of it, maintaining valid SQL syntax.

### Steps To Reproduce

Model 1:
```
-- models/ephemeral_model.sql
{{ config(materialized='ephemeral') }}
select 1 as id
```
Model 2:
```
-- models/foo.sql
select with (a AS 1, b AS 2, a + b) AS c
from {{ ref('ephemeral_model') }}
```

Run `dbt run -s foo`

dbt compiles the ephemeral model inside the WITH expression, producing invalid SQL:
```
create or replace table `cse-sandbox-319708`.`dbt_sprawira_bq_oauth`.`foo`
OPTIONS()
as (
select with __dbt__cte__ephemeral_model as (
select 1 as id
), (a AS 1, b AS 2, a + b) AS c
from __dbt__cte__ephemeral_model
)
```
This results in a syntax error in BigQuery.
```
04:55:41 Database Error in model foo (models/foo.sql)
Syntax error: Unexpected "(" at [15:45]
compiled code at target/run/my_new_project/models/foo.sql
```

I tested some modified code in BQ directly, and I was able to get this to work
```
-- The CTE is defined in a WITH clause first, then the SELECT uses the WITH expression and references the CTE.

WITH __dbt__cte__ephemeral_model AS (
select 1 as id
)
SELECT WITH (a AS 1, b AS 2, a + b) AS c
FROM __dbt__cte__ephemeral_model
```

### Relevant log output

```shell
as per above
```

### Environment

```markdown
- OS: macOS 10.15.7 (Catalina)
- Adapter: BigQuery
- dbt: Latest (Cloud)
```

### Additional Context

This seems to be similar to https://github.com/dbt-labs/dbt-core/issues/7350

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.