apache / apache/datafusion

Extract common regex compilation cache for reuse across regexp functions

Open
#23,675 0 comments 0 reactions 1 assignee Claimed by @andygrove View on GitHub
enhancement
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

## Is your feature request related to a problem or challenge?

While reviewing #23540 (optimizing `regexp_instr`), it was noted that the new `RegexCache` introduced there overlaps with the existing `compile_and_cache_regex` helper in `datafusion/functions/src/regex/mod.rs`.

The version added in #23540 is a superset: in addition to the `HashMap` keyed by `(pattern, flags)`, it memoizes the previous row's pattern (`last`) so that the common case of a literal pattern skips the hash lookup entirely on every row after the first. This gave a meaningful part of the ~40% speedup in that PR.

Today only `regexp_instr` benefits from this fast path. The other regex functions (`regexp_count`, `regexp_match`, `regexp_replace`) still use the plain `compile_and_cache_regex` HashMap lookup.

## Describe the solution you'd like

Extract the `RegexCache` pattern from `regexp_instr` into `datafusion/functions/src/regex/mod.rs` as a shared, reusable cache, and migrate the other regex functions onto it so they get the same last-pattern memoization:

- `regexp_count`
- `regexp_match`
- `regexp_replace`

This would consolidate the two caching implementations into one and let all regex functions skip per-row hashing for literal patterns.

## Describe alternatives you've considered

Leave the two implementations separate. This keeps `mod.rs` unchanged but duplicates the caching logic and leaves the other functions without the faster path.

## Additional context

Follow-on from #23540. Suggested by @alamb in review:
> Maybe we could (as a follow on PR) extract the common regexp caching functionality -- I personally prefer this Cache pattern -- for use in the other functions with regular expressions

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.