intel / intel/hyperscan

\b with UTF-8

Open
#328 1 comment 2 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
5.5k
Forks
816
Avg merge
4d 21h
Merged PRs (30d)
2

Description

Hi,

is there a way in Hyperscan to find a pattern where `\b` is followed by a non-ASCII character (such as`\bö`) in the input text? According to my experience, `\be` matches where it should, but `\bö` does not. For instance,
- `\be` matches "_ e_" and does not match "_ xe_"
- `\bö` does not match "_ ö_", but it does "_ xö_"

I get the same result irrespective of whether I use `HS_FLAG_UTF8` or not; `HS_FLAG_UCP` gives an error. I could not find anything about `\b` being incompatible with Unicode in the [documentation](https://intel.github.io/hyperscan/dev-reference/compilation.html); in fact, the only place the docs mention HS not supporting UTF8 or `\b` is in the [approximate matching](https://intel.github.io/hyperscan/dev-reference/compilation.html#approximate-matching) section, which is irrelevant to my use-case.

Thanks!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.