python / python/cpython

Data race in list.sort() on no-gil build

未关闭
#154,756 0 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

interpreter-core topic-free-threading type-bug
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Bug report

Bug description:
Summary

As reported in #153852 (TSAN0014), list.sort() writes each slot of the array with a plain, non-atomic store. When another thread concurrently accesses the same list, there is a data race between the in-place sort and the reader, which will be reported by TSAN.

How to reproduce

First we need to enable thread-sanitizer

  • ./configure --disable-gil --with-thread-sanitizer
  • TSAN_OPTIONS="halt_on_error=1 symbolize=1 history_size=4" ./python repro.py

Then run the following repro.py

import sys, threading

size, rounds, num_threads = 2000, 1500, 32
SCRAMBLED = sorted(range(size), key=lambda x: (x * 2654435761) & 0xFFFFFFFF)
global_list = list(SCRAMBLED)
enter, leave = threading.Barrier(num_threads + 1), threading.Barrier(num_threads + 1)

def reader():
    for _ in range(rounds):
        enter.wait()
        for _x in global_list:
            pass
        leave.wait()

def main_sorter():
    for _ in range(rounds):
        global_list[:] = SCRAMBLED
        enter.wait()
        global_list.sort()
        leave.wait()

ts = [threading.Thread(target=reader) for _ in range(num_threads)]
for t in ts: 
    t.start()
main_sorter()
for t in ts: 
    t.join()

Finally we could see the log from TSan

WARNING: ThreadSanitizer: data race (pid=3143552)
  Write of size 8 at 0xffffb6b3c018 by main thread:
    #0 binarysort Objects/listobject.c:1918 (python+0x1c64bc)
CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs
  • gh-154572

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先使用列出的 no-GIL 和 ThreadSanitizer 配置运行 repro.py,然后检查第 1918 行 binarysort 附近的 Objects/listobject.c。将 race 报告与链接的 gh-154572 工作进行比较。当复现不再产生 ThreadSanitizer data-race 报告时,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
c, python
领域
operating-systems, testing-qa
Issue 类型
缺陷
难度
4/5
预计耗时
3-5 天
活跃度
停滞
描述清晰度
基本清楚
新手友好度
30/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。