cockroachdb / cockroachdb/cockroach

Optimizer fails to push aggregate functions through CROSS JOIN, causing full table scans instead of LIMITED SCAN on PRIMARY KEY

Open
#171,626 2 comments 0 reactions 0 assignees View on GitHub
C-enhancement O-community T-sql-queries X-blathers-triaged
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

**Is your feature request related to a problem? Please describe.**
Hi, CockroachDB developers, thanks for reading my report. I found a missed optimization in the optimizer.
When computing aggregate functions (`MAX/MIN`) from multiple tables in a `CROSS JOIN`, the optimizer materializes the full cartesian product before aggregation, losing the `LIMITED SCAN` optimization available on **`PRIMARY KEY`** indexes.

```sql
CREATE TABLE t0 (c0 INT PRIMARY KEY);
CREATE TABLE t1 (c0 INT PRIMARY KEY);
INSERT INTO t0 SELECT * FROM generate_series(1, 1000000);
INSERT INTO t1 SELECT * FROM generate_series(1, 1000);

-- Since t0 is a PRIMARY KEY, MAX(t0.c0) can be obtained via LIMITED SCAN
SELECT MAX(t0.c0) FROM t0; -- 2ms
-- The optimizer cannot push the LIMIT through the CROSS JOIN, forcing FULL SCAN
SELECT MAX(t0.c0), MAX(t1.c0) FROM t0, t1; -- 3.796s

EXPLAIN SELECT MAX(t0.c0) FROM t0;
distribution: local
vectorized: true
• group (scalar)

└── • revscan
missing stats
table: t0@t0_pkey
spans: LIMITED SCAN
limit: 1

EXPLAIN SELECT MAX(t0.c0), Max(t1.c0) FROM t0, t1;
distribution: local
vectorized: true
• render

└── • group (scalar)

└── • cross join

├── • scan
│ missing stats
│ table: t0@t0_pkey
│ spans: FULL SCAN

└── • scan
missing stats
table: t1@t1_pkey
spans: FULL SCAN
```

**Describe the solution you'd like**
Apply aggregate splitting/pushdown optimization, which would enable LIMITED SCAN on PRIMARY KEY for MAX/MIN operations.

**Additional context**
This is a common optimization pattern. DBMS, such as MySQL, MariaDB, and Percona, can handle this by pushing aggregates through cross joins.
Here, `t0` contains 1 million rows, `t1` contains 100 million rows.

Image

Jira issue: CRDB-64749

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.