jessepollak / jessepollak/urlmatch

[Feature Request] Add method to compile match regex

Open
#8 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
40
Forks
6
PR merge metrics
No merged PRs in 30d

Description

First of all, thanks for creating such a nifty package! I was looking for something that does exactly this.
Coming to the main point, I'll be using the matching in a repetitive manner (inside a loop), so it would be nice to have this be fast.
I've been experimenting with the code and compiling the regex that is used for `.search(url)` gives a huge performance gain:
```python
from timeit import timeit

pat = "https://*.example.org"
url = "https://sub.example.org/abcd"

compiled_pat = compile_urlmatch(pat, path_required=False, fuzzy_scheme=True)

# timeit runs 1e6 iterations of the supplied code by default
t1 = timeit("urlmatch(pat, url, path_required=False, fuzzy_scheme=True)", globals=globals())
t2 = timeit("bool(compiled_pat.search(url))", globals=globals())

print(f"Vanilla urlmatch: {t1:.4f}s")
print(f"Compiled pattern: {t3:.4f}s")
```
Output:
```
Vanilla urlmatch: 11.4504s
Compiled pattern: 0.7760s
```

As you can see, compiling the regex beforehand gives us more than an order of magnitude improvement in performance, since the regex doesn't have to be constructed every time.
The `compile_urlmatch` function you see above is just a slight modification of the default `urlmatch` - it returns the compiled regex instead of immediately doing the search.
```python
def compile_urlmatch(match_pattern, **kwargs):
if isinstance(match_pattern, str):
match_pattern = map(str.strip, match_pattern.split(','))

regex = "({})".format("|".join(map(
lambda x: parse_match_pattern(x, **kwargs), match_pattern)))
if not regex: # not sure now to handle this for now, ValueError seems appropriate
raise ValueError
return re.compile(regex) # main change
```

To be able to implement this, I had to modify `__init__.py` to also export `parse_match_pattern` to the end user. So it would be nice if this could be upstreamed. You could either export just this function, or bake in the `compile_urlmatch` to the package itself.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the existing urlmatch implementation and __init__.py, especially parse_match_pattern and the public urlmatch entry point. Compare the proposed compile_urlmatch behavior with the current matching path, then verify that the chosen public API supports repeated searches without rebuilding the regex and preserves the documented options.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
web-dev
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.