apache / apache/datafusion

[substrait] Scalar subquery in select not supported

Open
#18,066 2 comments 0 reactions 2 assignees Claimed by @bvolpato View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

For certain queries with subqueries, DataFusion optimizes them as a LEFT join without condition (`ScalarSubqueryToJoin`).

A recent addition to require conditions for joins (https://github.com/apache/datafusion/pull/15334) has added the validation that causes problems in this type of query now:

```
---- cases::roundtrip_logical_plan::scalar_subquery_in_select stdout ----
Error: Plan("join condition should not be empty")
```

### To Reproduce

```sql
SELECT a, (SELECT MAX(b) FROM data2) as max_b FROM data
```
Returns error:

> Error: Plan("join condition should not be empty")

### Expected behavior

`Boolean(true)` in the filter:

```
Projection: data.a, max(data2.b) AS max_b
Left Join:
TableScan: data projection=[a]
Aggregate: groupBy=[[]], aggr=[[max(data2.b)]]
TableScan: data2 projection=[b], partial_filters=[Boolean(true)]
```

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.