cockroachdb / cockroachdb/cockroach

colexec: emit batches with large rows early

Open
#109,309 21 comments 0 reactions 0 assignees View on GitHub
branch-master C-test-failure O-roachtest O-robot O-rsg P-3 T-sql-queries X-nostale
Dominant language
Go
Stars
32.5k
Forks
4.1k
PR merge metrics
PR metrics pending

Description

Fix the vectorized engine to emit batches eagerly in case they have wide rows:
- merge joiner
- cross joiner
- vectorized `concat_agg` when used as window function.

Some reproductions

```sql
CREATE TABLE l (l_k INT, l_v STRING, INDEX (l_k) STORING (l_v));
CREATE TABLE r (r_k INT, r_v STRING, INDEX (r_k) STORING (r_v));
INSERT INTO l SELECT i // 100, repeat('l', 60000) FROM generate_series(0, 899) AS g(i);
INSERT INTO r SELECT i // 10, repeat('r', 60000) FROM generate_series(0, 99) AS g(i);
-- sizes in the next two queries can be tweaked for any target memory usage
INSERT INTO l SELECT 100000, repeat('l', 1000000) FROM generate_series(0, 99) AS g(i);
INSERT INTO r SELECT 100000, repeat('l', 1000000) FROM generate_series(0, 19) AS g(i);
ANALYZE l;
ANALYZE r;
EXPLAIN ANALYZE (VERBOSE) SELECT length(l_v), length(r_v) FROM l INNER MERGE JOIN r ON l_k = r_k;
```

```sql
CREATE TABLE t (s STRING);
INSERT INTO t SELECT repeat('a', 50) FROM generate_series(1, 1000);
ANALYZE t;
EXPLAIN ANALYZE SELECT concat_agg(t1.s ORDER BY t1.s DESC) FROM t AS t1, t AS t2;
```

Jira issue: CRDB-30864

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.