Data race in the `codec.errors` setter (`codecctx_errors_set`) on a shared stateful codec
还没有人认领这个 Issue。
- 主要语言
- Python
- 星标
- 77.2k
- 派生
- 35.9k
- PR 合并指标
- PR 指标待抓取
描述
Bug report
Bug description:
Bug description:
In the free-threaded build, the codecctx_errors_set decrefs the old handler(ERROR_DECREF(self->errors)) and stores the new one (self->errors = cb) without any synchronization
For any handler name other than the strict/ignore/replace sentinels (e.g. backslashreplace), self->errors holds a real refcounted PyUnicode.
The decref-then-assign is not atomic, so concurrent codec.errors = drops references incorrectly and can free a handler while it is still referenced.
Reproducer:
import codecs
import random
from threading import Thread
shared = codecs.getincrementalencoder('shift_jis')()
NAMES = ['backslashreplace', 'xmlcharrefreplace', 'namereplace']
def thread1():
for _ in range(20000):
try:
shared.errors = random.choice(NAMES)
except Exception:
pass
if __name__ == "__main__":
threads = [Thread(target=thread1) for _ in range(8)]
for t in threads: t.start()
for t in threads: t.join()
TSAN Report:
==================
WARNING: ThreadSanitizer: data race (pid=3576762)
Read of size 8 at 0x7fffb65a1cc0 by thread T2:
#0 codecctx_errors_set /cpython/./Modules/cjkcodecs/multibytecodec.c:194:5
#1 getset_set /cpython/Objects/descrobject.c:250:16
#2 _PyObject_GenericSetAttrWithDict /cpython/Objects/object.c:2049:19
#3 PyObject_GenericSetAttr /cpython/Objects/object.c:2120:12
#4 PyObject_SetAttr /cpython/Objects/object.c:1533:15
#5 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:12146:27
...
Previous write of size 8 at 0x7fffb65a1cc0 by thread T1:
#0 codecctx_errors_set /cpython/./Modules/cjkcodecs/multibytecodec.c:195:18
#1 getset_set /cpython/Objects/descrobject.c:250:16
#2 _PyObject_GenericSetAttrWithDict /cpython/Objects/object.c:2049:19
#3 PyObject_GenericSetAttr /cpython/Objects/object.c:2120:12
#4 PyObject_SetAttr /cpython/Objects/object.c:1533:15
#5 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:12146:27
...
SUMMARY: ThreadSanitizer: data race /cpython/./Modules/cjkcodecs/multibytecodec.c:194:5 in codecctx_errors_set
==================
CPython versions tested on:
CPython main branch
Operating systems tested on:
Linux
Linked PRs
- gh-153000
贡献指南
从这里开始
- 先读完整个 Issue,再读项目的贡献指南。
- 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
- Fork 仓库,在一个分支上完成修改。
- 提交 Pull Request,并在描述里引用这个 Issue 编号。
调研方向
从 Modules/cjkcodecs/multibytecodec.c 中的 codecctx_errors_set 开始,然后在带有 ThreadSanitizer 的 free-threaded build 下运行提供的多线程 reproducer。将行为与链接的 PR gh-153000 进行比较;当并发的 codec.errors 赋值不再产生报告的 race 或错误的 handler 生命周期行为时,即表示完成。
由索引模型根据 Issue 内容生成。
评估
- 技术栈
- c, python
- 领域
- backend
- Issue 类型
- 缺陷
- 难度
- 4/5
- 预计耗时
- 3-5 天
- 活跃度
- 停滞
- 描述清晰度
- 描述清楚
- 新手友好度
- 30/100