Speed up multiline regexes anchored by `^`.
未關閉
還沒有人認領這個 Issue。
extension-modules
performance
topic-regex
type-feature
- 主要語言
- Python
- 星號
- 77.2k
- 分支
- 35.9k
- PR 合併指標
- PR 指標待擷取
描述
Feature or enhancement
Proposal:
I noticed that for regexes of the form
regex = re.compile("^foo", re.MULTILINE)
regex.search(...)
there's a character by character loop calling SRE(match) every iteration. That's significantly slower than the regex
regex = re.compile("foo...")
regex.search(...)
which does a special prefix scan for "foo" without having to call SRE(match) on each character:
I would expect ^foo and foo to have more or less identical performance. A simple patch like this fixes that.
diff --git a/Modules/_sre/sre_lib.h b/Modules/_sre/sre_lib.h
index df377905bfa..70de4cccefd 100644
--- a/Modules/_sre/sre_lib.h
+++ b/Modules/_sre/sre_lib.h
@@ -1855,6 +1855,18 @@ SRE(search)(SRE_STATE* state, SRE_CODE* pattern)
return 0;
}
while (status == 0 && ptr < end) {
+ if (pattern[0] == SRE_OP_AT &&
+ pattern[1] == SRE_AT_BEGINNING_LINE &&
+ (void*) ptr > state->beginning &&
+ !SRE_IS_LINEBREAK((int) ptr[-1]))
+ {
+ /* fast-forward to the next newline character */
+ while (ptr < end && !SRE_IS_LINEBREAK((int) *ptr)) {
+ ptr++;
+ }
+ if (ptr >= end) {
+ return 0;
+ }
+ }
ptr++;
RESET_CAPTURE_GROUP();
TRACE(("|%p|%p|SEARCH\n", pattern, ptr));
Has this already been discussed elsewhere?
This is a minor feature, which does not need previous discussion elsewhere
Links to previous discussion of this feature:
No response
Linked PRs
- gh-148778
- gh-152339
貢獻指南
從這裡開始
- 先讀完整個 Issue,再讀專案的貢獻指南。
- 在 Issue 下留言說明你要接手 —— 這能避免兩個人做同樣的事。
- Fork 儲存庫,在一個分支上完成修改。
- 送出 Pull Request,並在描述裡引用這個 Issue 編號。
研究方向
先查看連結的 PR,然後閱讀 Modules/_sre/sre_lib.h 中的 SRE 搜尋迴圈,尤其是其中參照的前綴掃描程式碼。重現多行 ^foo 搜尋和未錨定的 foo 搜尋,並比較它們的效能。在不改變正規表示式行為的情況下,如果錨定情況的速度有可測量的接近,即視為完成。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- python
- 領域
- performance
- Issue 類型
- 功能
- 難度
- 4/5
- 預估耗時
- 3-5 天
- 活躍度
- 停滯
- 描述清晰度
- 描述清楚
- 新手友好度
- 35/100