intel / intel/hyperscan

Why chimera sometimes need scan the whole buffer with an anchored regex ?

Open
#139 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
5.5k
Forks
816
Avg merge
4d 21h
Merged PRs (30d)
2

Description

Hi,
Chimera sometimes need scan the whole buffer when a regular expression contains greedy pattern, for example, use a very simple pattern `e2*e` to match `a4444444444444444444444444444444444444444444444444444444444a`. vermSearchAligned and vermUnalign in vermicelli_sse.h will read every char, but this seems no necessary, it should failed at the first char, and 2 is not 4 obviously. When pattern changed to anchored "\Ae2*e", scanDoubleFast in noodle engine will read every char too. Both make the performance very poor. Imagine there is millions of 4.
Is this a bug or designed, and is there any way to improve it? Thanks.

Contributor guide

Open the contributing guide

Research direction

Reproduce the greedy-pattern case with `e2*e` and the anchored `\Ae2*e` example against a long buffer of `4` characters. Inspect `vermicelli_sse.h`, especially `vermSearchAligned` and `vermUnalign`, then trace `scanDoubleFast` in the noodle engine. Done means establishing whether the full-buffer scans are intended and identifying a justified performance improvement if they are not.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
performance, search
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.