apache / apache/datafusion

Support LATERAL subqueries with HAVING clause

Open
#21,198 1 comment 1 reaction 1 assignee Claimed by @neilconway View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

This currently doesn't work. The issue:

* For normal LATERAL subqueries without HAVING (but with COUNT(*) or a similar agg), we rewrite the LATERAL subquery into a grouped agg subquery and LEFT JOIN against it. We add a `CASE ... WHEN __always_true` construct to convert unmatched outer rows from `NULL` to the agg's default value (0 for count).
* When the LATERAL subquery has a HAVING clause, there's another possibility: an outer row might be unmatched because the entire group was excluded because it failed the HAVING clause. So the correct behavior is to elide the entire outer row, not substitute a 0 for the NULL count value.

We can fix this by

* Pulling the HAVING clause out of the subquery and evaluating it over the subquery's results instead. This requires rewriting it in terms of the post-join schema.
* Arrange to omit rows from the final result set that don't satisfy the rewritten HAVING clause

This is similar to what we already do in `scalar_subquery_to_join`, except slightly different because we want to omit the row entirely, not return NULL.

### To Reproduce

_No response_

### Expected behavior

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.