python / python/cpython

JIT doesn't trace exec()-generated functions called from hot loops

Open
#149,564 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core topic-JIT type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

When a hot loop calls functions generated via exec(), the loop body gets traced but the callees don't. Each callee is individually cold (called ~60 times), but the call site is hot (5,000+ calls total across different callees).

Concrete case: tinygrad generates ~5,000 pattern matching functions at startup via exec(). They're called from a tight loop:

for _, match, _ in pats:
    if (ret := match(uop, ctx)) is not None: return ret

PYTHON_JIT=1 on 3.14.4 produces no improvement. The specializer works (LOAD_ATTR_SLOT fires correctly), but the JIT never compiles the callees.

Related: faster-cpython/ideas#738, #118093 (tier 2 entry at function calls).

It would be useful to either trace through exec()-generated callees at hot call sites, or detect aggregate hotness across callees sharing a call site.

Linked PRs
  • gh-149575

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the tinygrad pattern-matching loop with PYTHON_JIT=1 on Python 3.14.4, focusing on the exec()-generated callees and their shared hot call site. Compare the specializer's LOAD_ATTR_SLOT behavior with whether the callees are compiled; done means the reported hot-call workload shows JIT improvement through the generated functions.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
compilers, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.