python / python/cpython

Data race in the `codec.errors` setter (`codecctx_errors_set`) on a shared stateful codec

未关闭
#152,767 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

extension-modules topic-free-threading type-bug
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Bug report

Bug description:
Bug description:

In the free-threaded build, the codecctx_errors_set decrefs the old handler(ERROR_DECREF(self->errors)) and stores the new one (self->errors = cb) without any synchronization

https://github.com/python/cpython/blob/ecdef1773006529b0fea6639d0effeecbb41679c/Modules/cjkcodecs/multibytecodec.c#L170-L197

For any handler name other than the strict/ignore/replace sentinels (e.g. backslashreplace), self->errors holds a real refcounted PyUnicode.

The decref-then-assign is not atomic, so concurrent codec.errors = drops references incorrectly and can free a handler while it is still referenced.

Reproducer:

import codecs
import random
from threading import Thread
shared = codecs.getincrementalencoder('shift_jis')()
NAMES = ['backslashreplace', 'xmlcharrefreplace', 'namereplace']

def thread1():
    for _ in range(20000):
        try:
            shared.errors = random.choice(NAMES)
        except Exception:
            pass

if __name__ == "__main__":
    threads = [Thread(target=thread1) for _ in range(8)]
    for t in threads: t.start()
    for t in threads: t.join()

TSAN Report:

==================
WARNING: ThreadSanitizer: data race (pid=3576762)
  Read of size 8 at 0x7fffb65a1cc0 by thread T2:
    #0 codecctx_errors_set /cpython/./Modules/cjkcodecs/multibytecodec.c:194:5 
    #1 getset_set /cpython/Objects/descrobject.c:250:16 
    #2 _PyObject_GenericSetAttrWithDict /cpython/Objects/object.c:2049:19
    #3 PyObject_GenericSetAttr /cpython/Objects/object.c:2120:12
    #4 PyObject_SetAttr /cpython/Objects/object.c:1533:15
    #5 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:12146:27 
...

  Previous write of size 8 at 0x7fffb65a1cc0 by thread T1:
    #0 codecctx_errors_set /cpython/./Modules/cjkcodecs/multibytecodec.c:195:18
    #1 getset_set /cpython/Objects/descrobject.c:250:16 
    #2 _PyObject_GenericSetAttrWithDict /cpython/Objects/object.c:2049:19
    #3 PyObject_GenericSetAttr /cpython/Objects/object.c:2120:12 
    #4 PyObject_SetAttr /cpython/Objects/object.c:1533:15 
    #5 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:12146:27 
...

SUMMARY: ThreadSanitizer: data race /cpython/./Modules/cjkcodecs/multibytecodec.c:194:5 in codecctx_errors_set
==================
CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-153000

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 Modules/cjkcodecs/multibytecodec.c 中的 codecctx_errors_set 开始,然后在带有 ThreadSanitizer 的 free-threaded build 下运行提供的多线程 reproducer。将行为与链接的 PR gh-153000 进行比较;当并发的 codec.errors 赋值不再产生报告的 race 或错误的 handler 生命周期行为时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
c, python
领域
backend
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
描述清楚
新手友好度
30/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。