python / python/cpython

[PGO] Provide Pre-Generated Profiles

Open
#127,425 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

build type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

PGO (Profile-Guided Optimization) offers significant performance benefits for Python. However, the current approach has a drawback: it makes the builds non-reproducible.

This issue could be resolved if upstream provided pre-generated profiles (perhaps via CI pipelines). These profiles could then be included in the distributed tarball, similar to how Chromium handles it. By doing so, distributions wouldn't need to:

  1. Compile the source code twice.
  2. Perform profiling themselves.
    In addition to streamlining the process for maintainers, this would also help maintain package reproducibility—a key consideration for many distributions.

I am not sure, about the integration with GCC, since in my testing the compilation failed with a preset profile (but I didnt dig deeper into this yet.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files, tests, or entry points are identified. Start by investigating CPython's PGO build and CI pipeline, especially how GCC consumes a preset profile. Done means producing a reproducible profile in CI and including it in the distributed tarball without requiring downstream profiling or a second compilation.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.