apache / apache/datafusion

Support correlated scalar subquery without aggregation

Open
#16,137 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Is your feature request related to a problem or challenge?

This is perfectly valid use of scalar subquery without aggregation.
This should work in DataFusion

```sql
WITH src AS (
SELECT *
FROM (VALUES
(1, NULL, 'Europe'),
(2, 1, 'Warsaw'),
(3, 1, 'Paris')
) t(id, parent_id, name)
)
SELECT
id,
name,
(SELECT p.name FROM src p WHERE p.id = s.parent_id) AS parent_name
FROM src s;
```

Currently this fails with

```
Invalid (non-executable) plan after Analyzer
caused by
Error during planning: Correlated scalar subquery must be aggregated to return at most one row
```

### Describe the solution you'd like

_No response_

### Describe alternatives you've considered

_No response_

### Additional context

- https://github.com/apache/datafusion/issues/16134

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the SQL example from the issue and inspect the Analyzer error for correlated scalar subqueries. Trace the planning path that requires aggregation; done means the query executes successfully and returns each parent name without requiring aggregation, with coverage for the shown case.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.