duckdb / duckdb/duckdb

Full string match regex function doesn't prune row group

Open
#24,042 1 comment 0 reactions 0 assignees View on GitHub
PR submitted
Dominant language
C++
Stars
41.5k
Forks
3.8k
Avg merge
2d 7h
Merged PRs (30d)
515

Description

### What happens?

See repro for details.

### To Reproduce

```sql
memory D CREATE OR REPLACE TABLE r3 AS
SELECT
i::INTEGER AS id,
('k' || lpad(i::VARCHAR, 8, '0')) AS s
FROM range(0, 1000000) t(i)
ORDER BY id;

-- should scan only one row group
memory D EXPLAIN ANALYZE SELECT count(*) FROM r3 WHERE replace(s, 'z', 'z') = 'k00900000';
╭─ Summary ───────────╮
│ Total Time: 0.0107s │
╰─────────────────────╯
╭─ Ungrouped Aggregate ────────────────╮
│ 1 row 0µs │
╰───────────────────┒┎─────────────────╯
╭─ Table Scan ──────┚┖─────────────────╮
│ Table: memory.main.r3 │
│ Type: Sequential Scan │
│ Filters: │
│ "replace"(s, 'z', 'z') = 'k00900000' │
│ Row Groups Scanned: 9 / 9 │
│ 1 row 50.0ms │
╰──────────────────────────────────────╯

-- should scan only one row group
memory D EXPLAIN ANALYZE SELECT count(*) FROM r3 WHERE regexp_full_match(s, 'k00900000');
╭─ Summary ───────────╮
│ Total Time: 0.0101s │
╰─────────────────────╯
╭─ Ungrouped Aggregate ─────────────╮
│ 1 row 0µs │
╰─────────────────┒┎────────────────╯
╭─ Table Scan ────┚┖────────────────╮
│ Table: memory.main.r3 │
│ Type: Sequential Scan │
│ Filters: │
│ regexp_full_match(s, 'k00900000') │
│ Row Groups Scanned: 9 / 9 │
│ 1 row 40.0ms │
╰───────────────────────────────────╯
```

### OS:

macos

### DuckDB Version:

abb2fb974537a975ce03dd23266056606b1a0e0b

### DuckDB Client:

SQL

### Hardware:

_No response_

### Full Name:

dentiny

### Affiliation:

N/A

### Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?

- [x] Yes, I have

### Did you include all code required to reproduce the issue?

- [x] Yes, I have

### Did you include all relevant data sets for reproducing the issue?

Yes

Contributor guide

Open the contributing guide

Research direction

Start by running the supplied SQL reproduction in DuckDB and compare EXPLAIN ANALYZE output for replace and regexp_full_match. Investigate the row-group pruning path used by these predicates. Done means the equivalent full-string match query scans only the matching row group instead of 9 / 9 while returning the expected count.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, sql
Domain
database
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.