cockroachdb / cockroachdb/cockroach
Optimizer fails to push aggregate functions through CROSS JOIN, causing full table scans instead of LIMITED SCAN on PRIMARY KEY
- Dominant language
- Go
- Stars
- 32.5k
- Forks
- 4.1k
- PR merge metrics
- PR metrics pending
Description
**Is your feature request related to a problem? Please describe.**
Hi, CockroachDB developers, thanks for reading my report. I found a missed optimization in the optimizer.
When computing aggregate functions (`MAX/MIN`) from multiple tables in a `CROSS JOIN`, the optimizer materializes the full cartesian product before aggregation, losing the `LIMITED SCAN` optimization available on **`PRIMARY KEY`** indexes.
```sql
CREATE TABLE t0 (c0 INT PRIMARY KEY);
CREATE TABLE t1 (c0 INT PRIMARY KEY);
INSERT INTO t0 SELECT * FROM generate_series(1, 1000000);
INSERT INTO t1 SELECT * FROM generate_series(1, 1000);
-- Since t0 is a PRIMARY KEY, MAX(t0.c0) can be obtained via LIMITED SCAN
SELECT MAX(t0.c0) FROM t0; -- 2ms
-- The optimizer cannot push the LIMIT through the CROSS JOIN, forcing FULL SCAN
SELECT MAX(t0.c0), MAX(t1.c0) FROM t0, t1; -- 3.796s
EXPLAIN SELECT MAX(t0.c0) FROM t0;
distribution: local
vectorized: true
• group (scalar)
│
└── • revscan
missing stats
table: t0@t0_pkey
spans: LIMITED SCAN
limit: 1
EXPLAIN SELECT MAX(t0.c0), Max(t1.c0) FROM t0, t1;
distribution: local
vectorized: true
• render
│
└── • group (scalar)
│
└── • cross join
│
├── • scan
│ missing stats
│ table: t0@t0_pkey
│ spans: FULL SCAN
│
└── • scan
missing stats
table: t1@t1_pkey
spans: FULL SCAN
```
**Describe the solution you'd like**
Apply aggregate splitting/pushdown optimization, which would enable LIMITED SCAN on PRIMARY KEY for MAX/MIN operations.
**Additional context**
This is a common optimization pattern. DBMS, such as MySQL, MariaDB, and Percona, can handle this by pushing aggregates through cross joins.
Here, `t0` contains 1 million rows, `t1` contains 100 million rows.
Jira issue: CRDB-64749
Contributor guide
Assessment
This issue has not been assessed yet.