apache / apache/arrow-rs

Support Overriding the Regex Size Limit

Open
#5,989 2 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

**Describe the bug**

#### Code

```rust
use arrow::array::StringArray;
use arrow::compute::regexp_is_match_utf8;

fn main() {
let array = StringArray::from(vec![Some("foo")]);
let regex_array = StringArray::from(vec![Some(r#"\w{1000}"#)]);
let ret = regexp_is_match_utf8(&array, ®ex_array, None).unwrap();
println!("{:?}", ret);
}
```

#### Error

```rust
called `Result::unwrap()` on an `Err` value: ComputeError("Regular expression did not compile: CompiledTooBig(10485760)")
stack backtrace:
0: rust_begin_unwind

```

#### Reason

https://github.com/rust-lang/regex/blob/8856fe36ac7dc37989e6ffb26b5fc57189bae626/src/builders.rs#L50-L57
```rust
impl Default for Builder {
fn default() -> Builder {
let metac = meta::Config::new()
.nfa_size_limit(Some(10 * (1 << 20))) // Default 10MB
.hybrid_cache_capacity(2 * (1 << 20));
Builder { pats: vec![], metac, syntaxc: syntax::Config::default() }
}
}
```

#### Solution

Replace `Regex::new(pattern.as_str())` with `RegexBuilder::new(pattern.as_str()).size_limit(regex_size_limit)`, and `regex_size_limit` can be obtained as a function argument or an environment variable.

If the community approves of this solution, I will implement it.

**To Reproduce**

**Expected behavior**

**Additional context**

Contributor guide

Open the contributing guide

Research direction

Start at the compute::regexp_is_match_utf8 entry point shown in the reproduction and trace how patterns are compiled. Compare the current Regex::new path with the issue's proposed RegexBuilder size_limit approach, then verify that the reported large pattern no longer fails and that the chosen configuration path is covered by regression testing.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.