Support Overriding the Regex Size Limit
- Dominant language
- Rust
- Stars
- 3.6k
- Forks
- 1.3k
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 169
Description
**Describe the bug**
#### Code
```rust
use arrow::array::StringArray;
use arrow::compute::regexp_is_match_utf8;
fn main() {
let array = StringArray::from(vec![Some("foo")]);
let regex_array = StringArray::from(vec![Some(r#"\w{1000}"#)]);
let ret = regexp_is_match_utf8(&array, ®ex_array, None).unwrap();
println!("{:?}", ret);
}
```
#### Error
```rust
called `Result::unwrap()` on an `Err` value: ComputeError("Regular expression did not compile: CompiledTooBig(10485760)")
stack backtrace:
0: rust_begin_unwind
```
#### Reason
https://github.com/rust-lang/regex/blob/8856fe36ac7dc37989e6ffb26b5fc57189bae626/src/builders.rs#L50-L57
```rust
impl Default for Builder {
fn default() -> Builder {
let metac = meta::Config::new()
.nfa_size_limit(Some(10 * (1 << 20))) // Default 10MB
.hybrid_cache_capacity(2 * (1 << 20));
Builder { pats: vec![], metac, syntaxc: syntax::Config::default() }
}
}
```
#### Solution
Replace `Regex::new(pattern.as_str())` with `RegexBuilder::new(pattern.as_str()).size_limit(regex_size_limit)`, and `regex_size_limit` can be obtained as a function argument or an environment variable.
If the community approves of this solution, I will implement it.
**To Reproduce**
**Expected behavior**
**Additional context**
Contributor guide
Research direction
Start at the compute::regexp_is_match_utf8 entry point shown in the reproduction and trace how patterns are compiled. Compare the current Regex::new path with the issue's proposed RegexBuilder size_limit approach, then verify that the reported large pattern no longer fails and that the chosen configuration path is covered by regression testing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100