apertium / apertium/lttoolbox

Expand (some) ICU character classes in regex_compiler?

Open
#160 3 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
C++
Stars
21
Forks
26
PR merge metrics
No merged PRs in 30d

Description

At the very least, getting Lower and Upper ranges would be nice, so we could
```xml
\\p{Lu}\\p{Ll}+
```
and whatnot.

If we do the "simple" thing and just expand like ranges in https://github.com/apertium/lttoolbox/blob/acx-spaces/lttoolbox/regexp_compiler.cc we get quite a lot of transitions
– https://www.compart.com/en/unicode/category/Ll (probably unreliable source) claims 2155 lowercase letters. But maybe it's OK if we keep regexes in their own `` – more research needed.

Alternatively, could/should we do something like insert a special symbol and have fst_processor treat it specially? (any tools operating on the compiled fst's like lt-trim or lt-print|hfst-txt2fst|hfst-stuff would just have to treat it opaquely)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.