apache / apache/datafusion

Fuse multiple scalar-aggregate subqueries over the same source into a single scan

Open
#23,213 1 comment 0 reactions 1 assignee Claimed by @nathanb9 View on GitHub
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

**Problem**

When a query computes several uncorrelated scalar-aggregate subqueries over the same table, DataFusion scans that table once per subquery.

**Proposed rewrite**

When 2 or more such subqueries share one source, fuse them into a single scan + aggregate, pushing each subquery's predicate into a `FILTER (WHERE ...)` clause:

```sql
-- Before: two scans of t
SELECT (SELECT count(*) FROM t WHERE a < 10),
(SELECT avg(x) FROM t WHERE a >= 10);

-- After: one scan of t
SELECT count(*) FILTER (WHERE a < 10),
avg(x) FILTER (WHERE a >= 10)
FROM t;
```

The source filter becomes the OR of the branch predicates, and each scalar subquery is replaced by a reference to the merged aggregate.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.