apache / apache/datafusion

Support correlated UNNEST in subquery

Open
#11,773 2 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

I cannot use `UNNEST` inside a subquery:

```sql
> SELECT id, (SELECT * FROM UNNEST(arr) LIMIT 1) FROM
(
SELECT 1 id, array [1, 2] arr
UNION
SELECT 2 id, array [] arr
) rows;

check_analyzed_plan
caused by
Error during planning: Unsupported operator in the subquery plan.
```

Nor can I use `UNNEST` in a subquery on the right hand side of `EXISTS`:

```sql
> SELECT id FROM
(
SELECT 1 id, array [1, 2] arr
UNION
SELECT 2 id, array [] arr
) rows
WHERE EXISTS (SELECT 1 FROM UNNEST(arr));

check_analyzed_plan
caused by
Error during planning: Unsupported operator in the subquery plan.
```

### To Reproduce

The queries above can be used in `datafusion-cli`.

### Expected behavior

The queries above should ideally work like they do in Postgres (with some type annotations):

```sql
# SELECT id, (SELECT * FROM UNNEST(arr) LIMIT 1) FROM (SELECT 1 id, array [1, 2] arr UNION SELECT 2 id, array [] :: integer[] arr) rows;

id | unnest
----+--------
1 | 1
2 |
(2 rows)

# SELECT id FROM (SELECT 1 id, array [1, 2] arr UNION SELECT 2 id, array [] :: integer[] arr) rows WHERE EXISTS (SELECT 1 FROM UNNEST(arr));
id
----
1
(1 row)
```

### Additional context

Sorry if this is already tracked. It seems like there is a missing case in this function, and if this is a simple fix then I'd be happy to try to make a PR:

https://github.com/apache/datafusion/blob/ae2ca6a0e21b77bba1ac40ea6ee059e47d0791e0/datafusion/optimizer/src/analyzer/subquery.rs#L233

Thanks!

Contributor guide

Open the contributing guide

Research direction

Start at datafusion/optimizer/src/analyzer/subquery.rs around line 233, then reproduce both correlated UNNEST queries with datafusion-cli. Trace the unsupported subquery-plan case and compare the expected PostgreSQL results in the issue. Done means both queries plan and execute successfully, with regression coverage for the reported cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.