dbt-labs / dbt-labs/dbt-adapters
Ephemeral model injection causes syntax error when using WITH expression syntax (variable declaration)
- Dominant language
- Python
- Stars
- 233
- Forks
- 362
- Avg merge
- 3d 22h
- Merged PRs (30d)
- 9
Description
### Is this a new bug?
- [x] I believe this is a new bug
- [x] I have searched the existing issues, and I could not find an existing issue for this bug
### Which packages are affected?
- [x] dbt-adapters
- [ ] dbt-tests-adapter
- [ ] dbt-athena
- [ ] dbt-athena-community
- [x] dbt-bigquery
- [ ] dbt-postgres
- [ ] dbt-redshift
- [ ] dbt-snowflake
- [ ] dbt-spark
### Current Behavior
When an ephemeral model is referenced inside a model that uses BigQuery's WITH expression syntax (for local variable declarations, not CTEs), dbt incorrectly injects the ephemeral model definition inside the WITH expression block.
This leads to a compilation syntax error because the compiler confuses the WITH expression with a standard WITH CTE clause.
### Expected Behavior
The compiled SQL should correctly handle the WITH expression and inject the ephemeral model as a CTE outside of it, maintaining valid SQL syntax.
### Steps To Reproduce
Model 1:
```
-- models/ephemeral_model.sql
{{ config(materialized='ephemeral') }}
select 1 as id
```
Model 2:
```
-- models/foo.sql
select with (a AS 1, b AS 2, a + b) AS c
from {{ ref('ephemeral_model') }}
```
Run `dbt run -s foo`
dbt compiles the ephemeral model inside the WITH expression, producing invalid SQL:
```
create or replace table `cse-sandbox-319708`.`dbt_sprawira_bq_oauth`.`foo`
OPTIONS()
as (
select with __dbt__cte__ephemeral_model as (
select 1 as id
), (a AS 1, b AS 2, a + b) AS c
from __dbt__cte__ephemeral_model
)
```
This results in a syntax error in BigQuery.
```
04:55:41 Database Error in model foo (models/foo.sql)
Syntax error: Unexpected "(" at [15:45]
compiled code at target/run/my_new_project/models/foo.sql
```
I tested some modified code in BQ directly, and I was able to get this to work
```
-- The CTE is defined in a WITH clause first, then the SELECT uses the WITH expression and references the CTE.
WITH __dbt__cte__ephemeral_model AS (
select 1 as id
)
SELECT WITH (a AS 1, b AS 2, a + b) AS c
FROM __dbt__cte__ephemeral_model
```
### Relevant log output
```shell
as per above
```
### Environment
```markdown
- OS: macOS 10.15.7 (Catalina)
- Adapter: BigQuery
- dbt: Latest (Cloud)
```
### Additional Context
This seems to be similar to https://github.com/dbt-labs/dbt-core/issues/7350
Contributor guide
Assessment
This issue has not been assessed yet.