Speed up matching of case-insensitive character sets

未關閉
#152,054 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視

還沒有人認領這個 Issue。

評估

難度
4/5
預估耗時
3-5 天
新手友好度
35/100
Issue 類型
功能
描述清晰度
基本清楚
活躍度
停滯
技術堆疊
python
領域
performance

研究方向

先檢查連結的 PR gh-152055，接著檢視 SRE(count) 進入點以及 IN、IN_IGNORE、IN_UNI_IGNORE 和 IN_LOC_IGNORE opcode 的處理。使用未套用修補程式和已套用修補程式的建置執行提供的 pyperf 基準測試，並確認不區分大小寫的字元集重複基準測試有所改善，同時不改變維持不變的否定集案例。

由索引模型根據 Issue 內容生成。

描述

performance stdlib topic-regex type-feature

Feature or enhancement

Proposal:

A REPEAT_ONE over a case-insensitive character set — e.g. [a-z]+ with re.IGNORECASE — does not use the fast SRE(count) path. The compiled inner opcode is IN_IGNORE / IN_UNI_IGNORE / IN_LOC_IGNORE, none of which has a case in SRE(count). The case-sensitive SRE_OP_IN already has a fast case.

Adding the three IN_*_IGNORE cases to SRE(count) lets them scan inline.

Benchmark

Benchmark before after
[a-z]+ re.I|re.A (IN_IGNORE) 1.28 us 538 ns 2.38x
[a-z]+ re.I (IN_UNI_IGNORE) 1.41 us 711 ns 1.98x
[aeiou]+ re.I 1.35 us 706 ns 1.91x
[a-z0-9]+ re.I 1.31 us 696 ns 1.88x
[a-z0-9_]+ re.I 1.31 us 703 ns 1.86x
[a-z]+ re.L|re.I bytes (IN_LOC_IGNORE) 2.09 us 1.38 us 1.52x
findall [a-z]+ re.I 109 us 88.8 us 1.22x
findall [a-z_][a-z0-9_]* re.I 103 us 89.2 us 1.15x

[^0-9]+ re.I is unchanged — it has no cased members, so it stays a plain
IN (already fast).

benchmark script (pyperf) 
"""Benchmark: SRE(count) fast path for case-insensitive set repeats."""
import re
import pyperf

N = 100
MIXED   = ("aBcDeFgHiJkLmNoPqRsTuVwX" * N)[:N]
ALNUM   = ("aB3dE6gH9kLmN0pQrStUvWx1" * N)[:N]
WORD    = ("aB_dE_gH_kLmN_pQrStUvW_1" * N)[:N]
NODIGIT = ("aBcDeF gHiJkL!mNoPqR.sT?" * N)[:N]
BYTES   = MIXED.encode("latin1")

SCANS = [
    ("scan_alpha_uni",  re.compile(r"[a-z]+",    re.I),        MIXED),
    ("scan_alpha_asc",  re.compile(r"[a-z]+",    re.I | re.A), MIXED),
    ("scan_alnum_uni",  re.compile(r"[a-z0-9]+", re.I),        ALNUM),
    ("scan_word_uni",   re.compile(r"[a-z0-9_]+",re.I),        WORD),
    ("scan_neg_uni",    re.compile(r"[^0-9]+",   re.I),        NODIGIT),
    ("scan_vowels_uni", re.compile(r"[aeiou]+",  re.I),        "aAeEiIoOuU" * (N // 10)),
    ("scan_alpha_loc",  re.compile(rb"[a-z]+",   re.L | re.I), BYTES),
]
DOC = ("The Quick Brown Fox jumps over 12 Lazy Dogs near IP 10_0_0_1 and Node7. " * 50)
FINDS = [
    ("find_words_ci", re.compile(r"[a-z]+",           re.I), DOC),
    ("find_ident_ci", re.compile(r"[a-z_][a-z0-9_]*", re.I), DOC),
]

def make_scan(p, s):
    def run():
        assert p.match(s) is not None
    return run

runner = pyperf.Runner()
for name, p, s in SCANS:
    runner.bench_func(name, make_scan(p, s))
for name, p, s in FINDS:
    runner.bench_func(name, (lambda p, s: lambda: p.findall(s))(p, s))

Run under the unpatched and patched builds, then
python -m pyperf compare_to before.json after.json --table.

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs
  • gh-152055
主要語言
Python
星號
77.2k
分支
36k
平均合併
1 天 9 小時
30 天內合併 PR
558

貢獻指南

開啟貢獻指南

從這裡開始

  1. 先讀完整個 Issue，再讀專案的貢獻指南。
  2. 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
  3. Fork 儲存庫，在一個分支上完成修改。
  4. 送出 Pull Request，並在描述裡引用這個 Issue 編號。

python/cpython 的其他 Issue

查看 python/cpython 的全部 Issue

相似的 Issue

更多 Python Issue

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。