python / python/cpython

CI: UBSan runs GIL-only and never against the JIT; two configurations already shipped by distros are unsanitized

Đang mở
#154,927 2 bình luận 0 reaction 0 người được giao Xem trên GitHub

Chưa có ai nhận issue này.

infra type-feature
Ngôn ngữ chính
Python
Star
77.2k
Fork
35.9k
Chỉ số merge pull request
Chỉ số pull request đang chờ

Mô tả

Feature or enhancement

Proposal

The UBSan job in .github/workflows/build.yml runs against exactly one configuration: GIL-enabled, no JIT. Two configurations that CI already builds — and that distributions already ship — are never sanitized. Both are cheap to add, and I have measured what each would have caught.

1. UBSan × free-threading: true

The sanitizer matrix pairs TSan with free-threading: {false, true} but UBSan only with false:

        free-threading:
        - false
        - true
        sanitizer:
        - TSan
        include:
        - check-name: Undefined behavior
          sanitizer: UBSan
          free-threading: false

So files that only compile under Py_GIL_DISABLED are never built under UBSan. That is not a hypothetical gap: Python/uniqueid.c:185 shifted a negative per-thread refcount delta left, which is UB, and it fired from interpreter startup and from every GC cycle. Over the full suite with all suppressions disabled:

UB reports test failures failing files
before 1762 529 61
after #154915 0 0 0

Once #154915 lands, this configuration is green, so adding it is a regression guard rather than a source of new work.

2. UBSan × --enable-experimental-jit

jit.yml builds the JIT in several configurations and tail-call.yml builds --with-tail-call-interp, but neither is ever combined with a sanitizer. The JIT's own C code — the stencil patcher in Python/jit.c, and the tier-2 optimiser in Python/optimizer*.c which only compiles under -D_Py_TIER2=1 — is therefore never sanitized in CI.

This one has already proven productive when done by hand: gh-139269 (an unaligned uint64_t store in Python/jit.c's patch_* functions, which segfaulted release builds) was found by building --enable-experimental-jit with -fsanitize=address,undefined, and gh-142476 was an ASan-found leak in allocate_executor.

I ran the full test suite against a machine-code JIT build under UBSan, with every entry in Tools/ubsan/suppressions.txt disabled: clean, zero reports, 51,459 tests across 493 files, 4m50s. Verified non-vacuous — _testinternalcapi.get_jit_backend() returns jit and a hot loop produces real machine-code ranges, so this was the copy-and-patch JIT rather than a silent fallback to the uop interpreter.

Since it is already green, this is also a pure regression guard.

Two smaller findings from the same sweep

  • local-bounds is enabled nowhere. --with-undefined-behavior-sanitizer injects plain -fsanitize=undefined, and local-bounds is not in that group even though it detects genuine UB. I built with it on and ran the full suite: zero findings, including no false positives from the trailing variable-length array idiom, which was the obvious risk. It looks free to add.
  • float-divide-by-zero (also not in the group) yields exactly one hit, Modules/expat/xmlparse.c:869, in vendored libexpat, which deliberately relies on IEEE-754 +inf and documents that in a comment. Not worth enabling without excluding expat.

On cost

The usual objection to widening the matrix is build time. For reference, a full PGO + ThinLTO + UBSan build — the heaviest combination I tried, including the instrumented build, the profile run, the rebuild and the ThinLTO link — took 9 min 43 s at --jobs=8. A plain UBSan build is far cheaper, and the JIT suite run above took under five minutes.

(That PGO/LTO configuration found nothing new, incidentally. It is worth noting only because --enable-optimizations and --with-lto appear in no workflow at all, while Arch, Debian and python-build-standalone all ship them. UBSan's checks are source-level, so PGO/LTO mostly changes which UB is reached; miscompilation from UB would show up as test failures rather than sanitizer reports, which is a different exercise.)

Linked PRs

  • #154915 — fixes the free-threaded UB, prerequisite for adding that matrix entry

Has this already been discussed elsewhere?

This is a follow-up to gh-148286.

Links to previous discussion of this feature:

  • gh-148286
  • gh-139269
  • gh-142476

Hướng dẫn đóng góp

Mở hướng dẫn đóng góp

Bắt đầu từ đâu

  1. Đọc hết issue, rồi đọc hướng dẫn đóng góp của dự án.
  2. Bình luận trên issue rằng bạn sẽ nhận — tránh hai người làm cùng một việc.
  3. Fork repository và làm thay đổi trên một nhánh.
  4. Mở pull request có tham chiếu số hiệu của issue.

Hướng nghiên cứu

Bắt đầu với .github/workflows/build.yml và so sánh ma trận UBSan của tệp này với các cấu hình workflow cho free-threading, JIT và tail-call. Kiểm tra điều kiện tiên quyết trong #154915, sau đó cập nhật cấu hình CI liên quan để UBSan bao phủ các build được đề xuất và vấn đề cần xem xét về local-bounds. Được xem là hoàn tất khi các job mới chạy thành công mà không có báo cáo sanitizer và vẫn giữ nguyên phạm vi bao phủ CI hiện có.

Do mô hình lập chỉ mục viết ra từ nội dung của issue.

Đánh giá

Công nghệ
github-actions, python
Lĩnh vực
build-system, ci-cd, testing-qa
Loại issue
Tính năng
Độ khó
3/5
Thời gian dự kiến
1-2 ngày
Mức độ hoạt động
Ít trao đổi
Độ rõ ràng
Khá rõ ràng
Mức phù hợp với người mới
62/100

Nhận issue mới trong hộp thư của bạn

Bản tóm tắt ngắn những issue GitHub phù hợp với người mới.