Speed up matching of case-insensitive character sets
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 35.9k
- PR 合并指标
- PR 指标待抓取
描述
Feature or enhancement
Proposal:
A REPEAT_ONE over a case-insensitive character set — e.g. [a-z]+ with re.IGNORECASE — does not use the fast SRE(count) path. The compiled inner opcode is IN_IGNORE / IN_UNI_IGNORE / IN_LOC_IGNORE, none of which has a case in SRE(count). The case-sensitive SRE_OP_IN already has a fast case.
Adding the three IN_*_IGNORE cases to SRE(count) lets them scan inline.
Benchmark
| Benchmark | before | after | |
|---|---|---|---|
[a-z]+ re.I|re.A (IN_IGNORE) |
1.28 us | 538 ns | 2.38x |
[a-z]+ re.I (IN_UNI_IGNORE) |
1.41 us | 711 ns | 1.98x |
[aeiou]+ re.I |
1.35 us | 706 ns | 1.91x |
[a-z0-9]+ re.I |
1.31 us | 696 ns | 1.88x |
[a-z0-9_]+ re.I |
1.31 us | 703 ns | 1.86x |
[a-z]+ re.L|re.I bytes (IN_LOC_IGNORE) |
2.09 us | 1.38 us | 1.52x |
findall [a-z]+ re.I |
109 us | 88.8 us | 1.22x |
findall [a-z_][a-z0-9_]* re.I |
103 us | 89.2 us | 1.15x |
[^0-9]+ re.I is unchanged — it has no cased members, so it stays a plain
IN (already fast).
benchmark script (pyperf)
"""Benchmark: SRE(count) fast path for case-insensitive set repeats."""
import re
import pyperf
N = 100
MIXED = ("aBcDeFgHiJkLmNoPqRsTuVwX" * N)[:N]
ALNUM = ("aB3dE6gH9kLmN0pQrStUvWx1" * N)[:N]
WORD = ("aB_dE_gH_kLmN_pQrStUvW_1" * N)[:N]
NODIGIT = ("aBcDeF gHiJkL!mNoPqR.sT?" * N)[:N]
BYTES = MIXED.encode("latin1")
SCANS = [
("scan_alpha_uni", re.compile(r"[a-z]+", re.I), MIXED),
("scan_alpha_asc", re.compile(r"[a-z]+", re.I | re.A), MIXED),
("scan_alnum_uni", re.compile(r"[a-z0-9]+", re.I), ALNUM),
("scan_word_uni", re.compile(r"[a-z0-9_]+",re.I), WORD),
("scan_neg_uni", re.compile(r"[^0-9]+", re.I), NODIGIT),
("scan_vowels_uni", re.compile(r"[aeiou]+", re.I), "aAeEiIoOuU" * (N // 10)),
("scan_alpha_loc", re.compile(rb"[a-z]+", re.L | re.I), BYTES),
]
DOC = ("The Quick Brown Fox jumps over 12 Lazy Dogs near IP 10_0_0_1 and Node7. " * 50)
FINDS = [
("find_words_ci", re.compile(r"[a-z]+", re.I), DOC),
("find_ident_ci", re.compile(r"[a-z_][a-z0-9_]*", re.I), DOC),
]
def make_scan(p, s):
def run():
assert p.match(s) is not None
return run
runner = pyperf.Runner()
for name, p, s in SCANS:
runner.bench_func(name, make_scan(p, s))
for name, p, s in FINDS:
runner.bench_func(name, (lambda p, s: lambda: p.findall(s))(p, s))
Run under the unpatched and patched builds, then
python -m pyperf compare_to before.json after.json --table.
Has this already been discussed elsewhere?
No response given
Links to previous discussion of this feature:
No response
Linked PRs
- gh-152055
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
首先检查链接的 PR gh-152055,然后检查 SRE(count) 入口点以及 IN、IN_IGNORE、IN_UNI_IGNORE 和 IN_LOC_IGNORE opcode 的处理。使用未打补丁和已打补丁的构建运行提供的 pyperf 基准测试,并确认不区分大小写的字符集重复基准测试有所改善,同时不改变保持不变的否定集情况。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- python
- 领域
- performance
- Issue 类型
- 功能
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 基本清楚
- 新手友好度
- 35/100