jessepollak / jessepollak/urlmatch
[Feature Request] Add method to compile match regex
- Dominant language
- Python
- Stars
- 40
- Forks
- 6
- PR merge metrics
- No merged PRs in 30d
Description
First of all, thanks for creating such a nifty package! I was looking for something that does exactly this.
Coming to the main point, I'll be using the matching in a repetitive manner (inside a loop), so it would be nice to have this be fast.
I've been experimenting with the code and compiling the regex that is used for `.search(url)` gives a huge performance gain:
```python
from timeit import timeit
pat = "https://*.example.org"
url = "https://sub.example.org/abcd"
compiled_pat = compile_urlmatch(pat, path_required=False, fuzzy_scheme=True)
# timeit runs 1e6 iterations of the supplied code by default
t1 = timeit("urlmatch(pat, url, path_required=False, fuzzy_scheme=True)", globals=globals())
t2 = timeit("bool(compiled_pat.search(url))", globals=globals())
print(f"Vanilla urlmatch: {t1:.4f}s")
print(f"Compiled pattern: {t3:.4f}s")
```
Output:
```
Vanilla urlmatch: 11.4504s
Compiled pattern: 0.7760s
```
As you can see, compiling the regex beforehand gives us more than an order of magnitude improvement in performance, since the regex doesn't have to be constructed every time.
The `compile_urlmatch` function you see above is just a slight modification of the default `urlmatch` - it returns the compiled regex instead of immediately doing the search.
```python
def compile_urlmatch(match_pattern, **kwargs):
if isinstance(match_pattern, str):
match_pattern = map(str.strip, match_pattern.split(','))
regex = "({})".format("|".join(map(
lambda x: parse_match_pattern(x, **kwargs), match_pattern)))
if not regex: # not sure now to handle this for now, ValueError seems appropriate
raise ValueError
return re.compile(regex) # main change
```
To be able to implement this, I had to modify `__init__.py` to also export `parse_match_pattern` to the end user. So it would be nice if this could be upstreamed. You could either export just this function, or bake in the `compile_urlmatch` to the package itself.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading the existing urlmatch implementation and __init__.py, especially parse_match_pattern and the public urlmatch entry point. Compare the proposed compile_urlmatch behavior with the current matching path, then verify that the chosen public API supports repeated searches without rebuilding the regex and preserves the documented options.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- web-dev
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100