Extract common regex compilation cache for reuse across regexp functions
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
## Is your feature request related to a problem or challenge?
While reviewing #23540 (optimizing `regexp_instr`), it was noted that the new `RegexCache` introduced there overlaps with the existing `compile_and_cache_regex` helper in `datafusion/functions/src/regex/mod.rs`.
The version added in #23540 is a superset: in addition to the `HashMap` keyed by `(pattern, flags)`, it memoizes the previous row's pattern (`last`) so that the common case of a literal pattern skips the hash lookup entirely on every row after the first. This gave a meaningful part of the ~40% speedup in that PR.
Today only `regexp_instr` benefits from this fast path. The other regex functions (`regexp_count`, `regexp_match`, `regexp_replace`) still use the plain `compile_and_cache_regex` HashMap lookup.
## Describe the solution you'd like
Extract the `RegexCache` pattern from `regexp_instr` into `datafusion/functions/src/regex/mod.rs` as a shared, reusable cache, and migrate the other regex functions onto it so they get the same last-pattern memoization:
- `regexp_count`
- `regexp_match`
- `regexp_replace`
This would consolidate the two caching implementations into one and let all regex functions skip per-row hashing for literal patterns.
## Describe alternatives you've considered
Leave the two implementations separate. This keeps `mod.rs` unchanged but duplicates the caching logic and leaves the other functions without the faster path.
## Additional context
Follow-on from #23540. Suggested by @alamb in review:
> Maybe we could (as a follow on PR) extract the common regexp caching functionality -- I personally prefer this Cache pattern -- for use in the other functions with regular expressions
Contributor guide
Assessment
This issue has not been assessed yet.