apache / apache/datafusion

Unparser with PostgreSqlDialect fails after Optimization Pass

Open
#21,066 2 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

The Unparser has a bug in the way that it treats certain PostgreSQL formatted queries, after optimization, which seems to be because `requires_derived_table_alias` is `true`:

I.e, optimizing then unparsing this query with postgresql dialect breaks:

```sql
WITH base AS (SELECT name, salary FROM t)
SELECT name,
CASE WHEN SUM(salary) > 0 THEN 1 ELSE 0 END AS x,
CASE WHEN SUM(salary) > 0 THEN SUM(salary) ELSE 0 END AS y
FROM base GROUP BY name
```

### To Reproduce

Here's a test we can add somewhere that fails:

```rust
#[tokio::test]
async fn test_cse_derived_projection_roundtrip() {
let ctx = SessionContext::new();
ctx.sql("CREATE TABLE t (name TEXT, salary DOUBLE) AS VALUES ('a', 1.0), ('b', 2.0)")
.await.unwrap().collect().await.unwrap();

let sql = "\
WITH base AS (SELECT name, salary FROM t) \
SELECT name, \
CASE WHEN SUM(salary) > 0 THEN 1 ELSE 0 END AS x, \
CASE WHEN SUM(salary) > 0 THEN SUM(salary) ELSE 0 END AS y \
FROM base GROUP BY name";

let df = ctx.sql(sql).await.unwrap();
let optimized = ctx.state().optimize(df.logical_plan()).unwrap();
let unparser = Unparser::new(&datafusion_sql::unparser::dialect::PostgreSqlDialect {});
let unparsed = unparser.plan_to_sql(&optimized).unwrap().to_string();

ctx.sql(&unparsed).await.unwrap_or_else(|e| {
panic!("Roundtrip failed.\n\nOriginal: {sql}\nUnparsed: {unparsed}\nError: {e}")
});
}
```

This fails with the following error message on `main`:

```
Roundtrip failed.

Original: WITH base AS (SELECT name, salary FROM t) SELECT name, CASE WHEN SUM(salary) > 0 THEN 1 ELSE 0 END AS x, CASE WHEN SUM(salary) > 0 THEN SUM(salary) ELSE 0 END AS y FROM base GROUP BY name
Unparsed: SELECT "base"."name", CASE WHEN "__common_expr_1" THEN 1 ELSE 0 END AS "x", CASE WHEN "__common_expr_1" THEN "sum(base.salary)" ELSE 0.0 END AS "y" FROM (SELECT (sum("base"."salary") > 0.0) AS "__common_expr_1", "base"."name", sum("base"."salary") FROM "t" AS "base" GROUP BY "base"."name") AS "derived_projection"
Error: Schema error: No field named base.name. Valid fields are derived_projection.__common_expr_1, derived_projection.name, derived_projection."sum(base.salary)".
```

### Expected behavior

This should parse correctly

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with the supplied test_cse_derived_projection_roundtrip reproduction and trace Unparser output for the PostgreSqlDialect after optimization. Inspect how requires_derived_table_alias affects the derived projection and verify the unparsed SQL can be parsed and executed successfully for the shown query.

Written by the indexing model from the issue text.

Assessment

Tech stack
postgresql, rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.