intel / intel/hyperscan

Low performance regex example

Open
#264 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
5.5k
Forks
816
Avg merge
4d 21h
Merged PRs (30d)
2

Description

I have tried to design a regular expression in a way that would result in a very large equivalent DFA, and found that the performance of HyperScan was significantly worse than RE2 even though RE2 had reverted to NFA mode. Based on the paper I expected that HyperScan would create multiple DFA's in order to perform better than an NFA implementation.

Is the RE2 NFA implementation just more efficient than HyperScan's DFA implementation in general, or is something wrong with how I set up my comparison?

The regular expression I used was:
```X((AA)*|(AAA)*|(AAAAA)*|(AAAAAAA)*|(AAAAAAAAAAA)*|(AAAAAAAAAAAAA)*|(AAAAAAAAAAAAAAAAA)*|(AAAAAAAAAAAAAAAAAAA)*|(AAAAAAAAAAAAAAAAAAAAAAA)*|(AAAAAAAAAAAAAAAAAAAAAAAAAAAAA)*)Y```

And in my benchmark I compile the regex and run against many lines of the form:
```XAAAAAA...AAAAAAY```
with between 100 and 1000 `A`'s.

I use block matching mode, and set the flags so that only a single match is needed. Before doing this match, I ran both RE2 and HyperScan regex's against a very long string in order to warm up the DFA state tables (in the case of RE2, this resulted in a OOM causing the system to revert to NFA mode).

In my benchmark, RE2 took 7 seconds and Hyperscan took 18 seconds.

When I tried a more normal-looking regular expression, I indeed find that HyperScan was much faster than RE2, so it doesn't seem to be a problem with compiler flags either.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.