Full string match regex function doesn't prune row group
- Dominant language
- C++
- Stars
- 41.5k
- Forks
- 3.8k
- Avg merge
- 2d 7h
- Merged PRs (30d)
- 515
Description
### What happens?
See repro for details.
### To Reproduce
```sql
memory D CREATE OR REPLACE TABLE r3 AS
SELECT
i::INTEGER AS id,
('k' || lpad(i::VARCHAR, 8, '0')) AS s
FROM range(0, 1000000) t(i)
ORDER BY id;
-- should scan only one row group
memory D EXPLAIN ANALYZE SELECT count(*) FROM r3 WHERE replace(s, 'z', 'z') = 'k00900000';
╭─ Summary ───────────╮
│ Total Time: 0.0107s │
╰─────────────────────╯
╭─ Ungrouped Aggregate ────────────────╮
│ 1 row 0µs │
╰───────────────────┒┎─────────────────╯
╭─ Table Scan ──────┚┖─────────────────╮
│ Table: memory.main.r3 │
│ Type: Sequential Scan │
│ Filters: │
│ "replace"(s, 'z', 'z') = 'k00900000' │
│ Row Groups Scanned: 9 / 9 │
│ 1 row 50.0ms │
╰──────────────────────────────────────╯
-- should scan only one row group
memory D EXPLAIN ANALYZE SELECT count(*) FROM r3 WHERE regexp_full_match(s, 'k00900000');
╭─ Summary ───────────╮
│ Total Time: 0.0101s │
╰─────────────────────╯
╭─ Ungrouped Aggregate ─────────────╮
│ 1 row 0µs │
╰─────────────────┒┎────────────────╯
╭─ Table Scan ────┚┖────────────────╮
│ Table: memory.main.r3 │
│ Type: Sequential Scan │
│ Filters: │
│ regexp_full_match(s, 'k00900000') │
│ Row Groups Scanned: 9 / 9 │
│ 1 row 40.0ms │
╰───────────────────────────────────╯
```
### OS:
macos
### DuckDB Version:
abb2fb974537a975ce03dd23266056606b1a0e0b
### DuckDB Client:
SQL
### Hardware:
_No response_
### Full Name:
dentiny
### Affiliation:
N/A
### Did you include all relevant configuration (e.g., CPU architecture, Linux distribution) to reproduce the issue?
- [x] Yes, I have
### Did you include all code required to reproduce the issue?
- [x] Yes, I have
### Did you include all relevant data sets for reproducing the issue?
Yes
Contributor guide
Research direction
Start by running the supplied SQL reproduction in DuckDB and compare EXPLAIN ANALYZE output for replace and regexp_full_match. Investigate the row-group pruning path used by these predicates. Done means the equivalent full-string match query scans only the matching row group instead of 9 / 9 while returning the expected count.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, sql
- Domain
- database
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100