intel / intel/hyperscan

Regex for unicode invisible control characters does not match all cases

Open
#342 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
5.5k
Forks
816
Avg merge
4d 21h
Merged PRs (30d)
2

Description

While using the regex `\p{Cf}`, which is expected to match invisible formatting indicators in Unicode, we found some inconsistent behavior. Working through this list, for example: https://www.compart.com/en/unicode/category/Cf

| Character | Type | Result |
|--|--|--|
| `\00AD` | Soft Hyphen | Match |
| `\0602` | Arabic Footnote Marker | No match |
| `\202A` | Left to Right Embedding | No match |
| `\200B` | Zero-width Space | No match |
| `\2064` | Invisible Plus | No match |

I can see that `\p{Cf}` is supported, but are there any rules governing which matches are valid? Is this expected behavior?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.