apache / apache/datafusion-comet
Add Java-generated Rust parity fixtures for regex crate upgrades
- Dominant language
- Scala
- Stars
- 1.3k
- Forks
- 373
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 198
Description
### What is the problem the feature request solves?
The `CometRegex` whitelist introduced by PR #5415 depends on the semantics and compilation behavior of the locked Rust `regex` crate.
The existing `CometRegexParitySuite` exercises the native kernel, but its admitted pattern corpus is relatively small. A future `regex` crate upgrade could change the result or compilation status of an admitted pattern without the current corpus detecting it.
This creates coupling between the plan-time admission rules and the specific `regex` crate version.
### Describe the potential solution
Generate a committed table of `(pattern, subject, expected)` fixtures using Java's `Pattern.matcher(subject).find()` and execute those fixtures directly in the Rust `rlike` unit tests.
The fixtures should cover:
- every construct currently admitted by `CometRegex`;
- concatenation and alternation combinations;
- capturing and non-capturing groups;
- character classes, ranges, negated classes, and escaped literals;
- greedy and counted quantifiers;
- empty-match cases;
- ASCII, non-ASCII, control-character, newline, and supplementary-code-point subjects;
- representative patterns near the conservative structural limits.
Document the JDK version and fixture-generation process. The committed Rust test should not require a JVM and should report the pattern, subject, expected result, and actual result on failure.
A Rust `regex` crate upgrade that changes the semantics or compilation status of an admitted fixture should cause this test to fail until the change is deliberately evaluated.
### Additional context
Follow-up from #5415 and #5351.
This tracks the Rust-side Java-generated fixture table requested during review of #5415: https://github.com/apache/datafusion-comet/pull/5415#pullrequestreview-5154502402
The review specifically highlighted the regex crate-version coupling.
Contributor guide
Research direction
Start with CometRegexParitySuite and the Rust rlike unit tests, then review PR #5415 and issue #5351 for the admission rules and review context. Done means committed Java-generated fixtures cover the listed constructs and subjects, Rust tests run without a JVM, failures show pattern, subject, expected, and actual values, and the JDK and generation process are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, rust
- Domain
- testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100