apache / apache/datafusion

Physical plan does not support logical expression Exists

Closed
#23,022 1 comment 1 reaction 1 assignee Claimed by @zyuiop View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

The following query (from the HotCRP open source software) fails to build a physical plan:

```sql
select (select group_concat(paperId, ' ', conflictType) from PaperConflict where contactId=1), exists (select * from PaperReview where contactId=1 and reviewType>0), exists (select * from PaperReview where requestedBy=1 and reviewType>0 and reviewType<=2 and contactId!=1), 0
```

The error is

```
DataFusion error: This feature is not implemented: Physical plan does not support logical expression Exists(Exists { subquery: , negated: false })
```

The reason is that some queries have subqueries in their projection, but the `decorrelate_predicate_subquery` optimizer step only accepts filter plans.

### To Reproduce

A smaller query is:

```sql
select exists (select paperId from PaperReview)
```

### Expected behavior

_No response_

### Additional context

Logical plan:

```
Projection: (), EXISTS () AS EXISTS (SELECT * FROM PaperReview WHERE contactId = 1 AND reviewType > 0), EXISTS () AS EXISTS (SELECT * FROM PaperReview WHERE requestedBy = 1 AND reviewType > 0 AND reviewType <= 2 AND contactId <> 1), Int64(0) AS 0
Subquery:
Aggregate: groupBy=[[]], aggr=[[group_concat(CAST(PaperConflict.paperId AS Utf8), Utf8(" "), CAST(PaperConflict.conflictType AS Utf8))]]
TableScan: PaperConflict projection=[paperId, conflictType], full_filters=[PaperConflict.contactId = Int32(1)]
Subquery:
TableScan: PaperReview projection=[paperId, reviewId, contactId, reviewType, requestedBy, reviewToken, reviewRound, reviewOrdinal, reviewBlind, reviewTime, reviewModified, reviewSubmitted, reviewAuthorSeen, timeDisplayed, timeApprovalRequested, reviewNeedsSubmit, reviewViewScore, rflags, timeRequested, timeRequestNotified, reviewAuthorModified, reviewNotified, reviewAuthorNotified, reviewEditVersion, reviewWordCount, s01, s02, s03, s04, s05, s06, s07, s08, s09, s10, s11, tfields, sfields], full_filters=[PaperReview.contactId = Int32(1), PaperReview.reviewType > Int8(0)]
Subquery:
TableScan: PaperReview projection=[paperId, reviewId, contactId, reviewType, requestedBy, reviewToken, reviewRound, reviewOrdinal, reviewBlind, reviewTime, reviewModified, reviewSubmitted, reviewAuthorSeen, timeDisplayed, timeApprovalRequested, reviewNeedsSubmit, reviewViewScore, rflags, timeRequested, timeRequestNotified, reviewAuthorModified, reviewNotified, reviewAuthorNotified, reviewEditVersion, reviewWordCount, s01, s02, s03, s04, s05, s06, s07, s08, s09, s10, s11, tfields, sfields], full_filters=[PaperReview.requestedBy = Int32(1), PaperReview.reviewType > Int8(0), PaperReview.reviewType <= Int8(2), PaperReview.contactId != Int32(1)]
EmptyRelation: rows=1
```

Plan for the smaller minimal example query:

```
Projection: EXISTS () AS EXISTS (SELECT paperId FROM PaperReview)
Subquery:
TableScan: PaperReview projection=[paperId]
EmptyRelation: rows=1
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.