python / python/cpython

Refactor perf trampoline proposal: split the assembly per architecture, auto-generate DWARF unwind data

未关闭
#149,800 5 条评论 1 个 reaction 已指派 0 人 在 GitHub 查看

还没有人认领这个 Issue。

interpreter-core type-refactor
主要语言
Python
星标
77.2k
派生
35.9k
PR 合并指标
PR 指标待抓取

描述

Feature or enhancement

Proposal:

Not a user visible feature per se but I still think it warrants discussion.

The problem:

I've been working on fixing up the RISC-V support for perf and also checking out adding s390x and ppc64le but the process is quite tedious, I'd say and the various connections around this infra are fragile, so I've been thinking of how to make the process a little bit easier and more maintainable/readable.

The steps to add perf trampoline support for a new CPU architecture currently:

The DWARF CFI instructions must mirror the assembly. This synchronization needs to be done manually and involves compiling a C equivalent of the trampoline, running readelf, and hand-translating the output into DWRF macros. Getting it wrong can produces silent failures, broken stack unwinding.

With x86-64, aarch64, and RISC-V there (RISC-V broken atm), when adding more things there, the assembly file is becoming bigger and less readable, elf_init_ehframe_perf() is becoming a wall of #ifdef blocks without much shared logic and so on.

Proposal1:

Split assembly files.

Replace the single Python/asm_trampoline.S with per-arch files:

  • Python/asm_trampoline_x86_64.S
  • Python/asm_trampoline_aarch64.S
  • Python/asm_trampoline_riscv64.S
  • (maybe in the future: asm_trampoline_ppc64le.S, asm_trampoline_s390x.S)

configure.ac is modified accordingly and selects which file to compile depending on arch. With each file self-contained, each arch can be reviewed, modified, tested independently. This will also require a few more lines in configure and careful handling of the MacOS case.

Proposal2 (followup):

Auto-generate DWARF data from compiled assembly.

A build-time script extracts the .eh_frame section from the compiled trampoline object and generates a header with the raw DWARF unwind data. At runtime, elf_init_ehframe_perf() copies this data and patches in the actual code address and size, replacing the current architecture-specific DWARF generation code entirely.

The JIT's Tools/jit/build.py already extracts DWARF CFI data from compiled objects to generate jit_unwind_info-.h for the GDB unwind path so something similar could be deployed here.

Another possibility would be a dependency on readelf, but I don't think that this is something desirable.

Pros and cons

Pros:

  • Adding a new architecture requires exactly one new file (the assembly) and a few lines in configure.ac. No DWARF knowledge needed (assuming the DWARF extraction script is robust :) ).
  • Eliminates possible bugs with syncronization of DWARF data (or move them to the extraction script, either way it should be simpler to deal with)
  • jit_unwind.c's elf_init_ehframe_perf shrinks to only the arch-independent code.

Cons

  • PYTHON_FOR_REGEN must be available when the trampoline assembly changes (same constraint as JIT stencil generation). That practically means that perf support will now depend on PYTHON_FOR_REGEN, making it unavailable for "clean" builds.
Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs
  • gh-149894
  • gh-150364
  • gh-150365
  • gh-157246

贡献指南

打开贡献指南

从这里开始

  1. 先读完整个 Issue,再读项目的贡献指南。
  2. 在 Issue 下留言说明你要接手 —— 这能避免两个人做同样的事。
  3. Fork 仓库,在一个分支上完成修改。
  4. 提交 Pull Request,并在描述里引用这个 Issue 编号。

调研方向

首先阅读 Python/asm_trampoline.S、Python/jit_unwind.c、Python/perf_jit_trampoline.c、configure.ac 和 Tools/jit/build.py,以了解当前特定于架构的汇编和 DWARF 生成方式。要视为完成,需要就按架构划分的 trampoline 和自动生成 unwind 数据达成一致并完成测试,但列出的链接 PRs 表明该提案已经在其他地方推进。

由索引模型根据 Issue 内容生成。

评估

技术栈
c, python
领域
build-system, performance
Issue 类型
功能
难度
5/5
预计耗时
一周以上
活跃度
停滞
描述清晰度
基本清楚
新手友好度
25/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。