byexamples / byexamples/byexample
Use the `regex` package in replacement of the built-in `re` module
- Dominant language
- Python
- Stars
- 67
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
**Describe the feature you'd like**
The idea is to replace the standard [re](https://docs.python.org/3/library/re.html) module with the third party [regex](https://pypi.org/project/regex/) module.
There are a few reasons to do it:
- the was not intention of [releasing the GIL](https://bugs.python.org/issue23690) by `re` while `regex` can do it (and it can be enforced with `concurrent=True`). The lack of multithreading support by `re` prevented to use threads instead of multi-processes in `byexample` which are, obviously, more expensive.
- `byexample` uses heavily the regex engine but all the regex used in a single run are different, unique. This means that the traditional cache of `re` is pointless because it disappears after each run (`byexample` is restarted and as any process the OS frees its memory). That means that in each run `byexample` needs to re-compile every single regex which it is very expensive. Pickling is pointless because currently `re` pickles only the expression and it compiles it when it loads the pickle so we don't save any time. `regex`, however, supports pickling the bytecode directly. **Note:** we should test how much we win with this.
- despite of been heavily optimized, the regex created by `byexample` may lead to a catastrophic collapse (endless high CPU usage). `re` does not support *atomic groups* or *possessive qualifiers* that could [reduce the impact of a catastrophic backtracking](https://www.regular-expressions.info/catastrophic.html) (see #16). `re` neither supports timeouts. `regex` in the other hand supports all of them.
Contributor guide
Assessment
This issue has not been assessed yet.