apache / apache/datafusion

Correctness issue with unparsing UNION & UNION ALL

Open
#24,446 2 comments 0 reactions 1 assignee Claimed by @Jeadie View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

Consider `a UNION ALL (b UNION a)`, that becomes a plan

```
Union
a
Distinct::All
Union
b
a
```

This will get incorrectly unparsed as `SELECT a UNION SELECT b UNION SELECT a`.

All union operations are now distinct, instead of correctly preserving the true per node nature. Root cause: traversing the inner `Distinct::All(Union)` sets `QueryBuilder::distinct_union = true` globally. All subsequent Unions are considered distinct during unpausing the query. A single, query-level boolean is insufficient to track `ast::SetQuantifier` for each UNION operation.

### To Reproduce

Add this example case to `datafusion/sql/tests/cases/plan_to_sql.rs` and rerun.
```
r#"SELECT j1_string FROM j1 UNION ALL (SELECT j2_string FROM j2 UNION SELECT j1_string FROM j1)"#,
```

### Expected behavior

**Actual**: the outer ALL is dropped.
```
SELECT a.x FROM a UNION SELECT b.x FROM b UNION SELECT a.x FROM a
```

**Expected**
```
SELECT a.x FROM a UNION ALL (SELECT b.x FROM b UNION SELECT a.x FROM a)
```

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.