python / python/cpython

CI: UBSan runs GIL-only and never against the JIT; two configurations already shipped by distros are unsanitized

未关闭
#154,927 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

infra type-feature
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Feature or enhancement

Proposal

The UBSan job in .github/workflows/build.yml runs against exactly one configuration: GIL-enabled, no JIT. Two configurations that CI already builds — and that distributions already ship — are never sanitized. Both are cheap to add, and I have measured what each would have caught.

1. UBSan × free-threading: true

The sanitizer matrix pairs TSan with free-threading: {false, true} but UBSan only with false:

        free-threading:
        - false
        - true
        sanitizer:
        - TSan
        include:
        - check-name: Undefined behavior
          sanitizer: UBSan
          free-threading: false

So files that only compile under Py_GIL_DISABLED are never built under UBSan. That is not a hypothetical gap: Python/uniqueid.c:185 shifted a negative per-thread refcount delta left, which is UB, and it fired from interpreter startup and from every GC cycle. Over the full suite with all suppressions disabled:

UB reports test failures failing files
before 1762 529 61
after #154915 0 0 0

Once #154915 lands, this configuration is green, so adding it is a regression guard rather than a source of new work.

2. UBSan × --enable-experimental-jit

jit.yml builds the JIT in several configurations and tail-call.yml builds --with-tail-call-interp, but neither is ever combined with a sanitizer. The JIT's own C code — the stencil patcher in Python/jit.c, and the tier-2 optimiser in Python/optimizer*.c which only compiles under -D_Py_TIER2=1 — is therefore never sanitized in CI.

This one has already proven productive when done by hand: gh-139269 (an unaligned uint64_t store in Python/jit.c's patch_* functions, which segfaulted release builds) was found by building --enable-experimental-jit with -fsanitize=address,undefined, and gh-142476 was an ASan-found leak in allocate_executor.

I ran the full test suite against a machine-code JIT build under UBSan, with every entry in Tools/ubsan/suppressions.txt disabled: clean, zero reports, 51,459 tests across 493 files, 4m50s. Verified non-vacuous — _testinternalcapi.get_jit_backend() returns jit and a hot loop produces real machine-code ranges, so this was the copy-and-patch JIT rather than a silent fallback to the uop interpreter.

Since it is already green, this is also a pure regression guard.

Two smaller findings from the same sweep

  • local-bounds is enabled nowhere. --with-undefined-behavior-sanitizer injects plain -fsanitize=undefined, and local-bounds is not in that group even though it detects genuine UB. I built with it on and ran the full suite: zero findings, including no false positives from the trailing variable-length array idiom, which was the obvious risk. It looks free to add.
  • float-divide-by-zero (also not in the group) yields exactly one hit, Modules/expat/xmlparse.c:869, in vendored libexpat, which deliberately relies on IEEE-754 +inf and documents that in a comment. Not worth enabling without excluding expat.

On cost

The usual objection to widening the matrix is build time. For reference, a full PGO + ThinLTO + UBSan build — the heaviest combination I tried, including the instrumented build, the profile run, the rebuild and the ThinLTO link — took 9 min 43 s at --jobs=8. A plain UBSan build is far cheaper, and the JIT suite run above took under five minutes.

(That PGO/LTO configuration found nothing new, incidentally. It is worth noting only because --enable-optimizations and --with-lto appear in no workflow at all, while Arch, Debian and python-build-standalone all ship them. UBSan's checks are source-level, so PGO/LTO mostly changes which UB is reached; miscompilation from UB would show up as test failures rather than sanitizer reports, which is a different exercise.)

Linked PRs

  • #154915 — fixes the free-threaded UB, prerequisite for adding that matrix entry

Has this already been discussed elsewhere?

This is a follow-up to gh-148286.

Links to previous discussion of this feature:

  • gh-148286
  • gh-139269
  • gh-142476

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

从 .github/workflows/build.yml 开始,将其 UBSan 矩阵与 free-threading、JIT 和 tail-call 的 workflow 配置进行比较。检查 #154915 中的前置条件,然后更新相关的 CI 配置,使 UBSan 覆盖提议的构建以及对 local-bounds 的考量。新作业能够成功运行且没有 sanitizer 报告,同时保留现有的 CI 覆盖范围,即表示完成。

由索引模型根据 Issue 内容生成。

评估

技术栈
github-actions, python
领域
build-system, ci-cd, testing-qa
Issue 类型
功能
难度
3/5
预计耗时
1-2 天
活跃度
冷清
描述清晰度
基本清楚
新手友好度
62/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。