python / python/cpython

Calling comprehensions always makes CALL_PY_EXACT_ARGS miss

Open
#93,827 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core type-feature
Dominant language
Python
Stars
77.2k
Forks
36k
PR merge metrics
PR metrics pending

Description

Consider the function

def f():
    return [x for x in range(2)]
for i in range(20):
    f() # warmup
from dis import dis
dis(f, adaptive=True)
dis(f)
  1           0 RESUME_QUICK             0

  2           2 LOAD_CONST               1 (<code object <listcomp> at 0x000001DA71743920, file "<stdin>", line 2>)
              4 MAKE_FUNCTION            0
              6 LOAD_GLOBAL_BUILTIN      1 (NULL + range)
             18 LOAD_CONST               2 (2)
             20 CALL_BUILTIN_CLASS       1
             30 GET_ITER
             32 CALL_PY_EXACT_ARGS       0
             42 POP_TOP
             44 LOAD_CONST               0 (None)
             46 RETURN_VALUE

Disassembly of <code object <listcomp> at 0x000001DA71743920, file "<stdin>", line 2>:
  2           0 RESUME_QUICK             0
              2 BUILD_LIST               0
              4 LOAD_FAST                0 (.0)
        >>    6 FOR_ITER_RANGE           4 (to 18)
             10 STORE_FAST__LOAD_FAST     1 (x)
             12 LOAD_FAST                1 (x)
             14 LIST_APPEND              2
             16 JUMP_BACKWARD_QUICK      6 (to 6)
        >>   18 RETURN_VALUE

That CALL_PY_EXACT_ARGS always misses at

DEOPT_IF(func->func_version != read_u32(cache->func_version), CALL);

because the function is created by MAKE_FUNCION so it will never have the same func_version as the previous time.

CALL_PY_EXACT_ARGS misses roughly 5% of the time in pyperformance, and this might decrease that a hair.

Maybe this is insignificant enough that it won't matter, but it would theoretically be nice to statically decide not to specialize these calls.

cc @markshannon

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the CALL_PY_EXACT_ARGS specialization for functions created by MAKE_FUNCTION, focusing on the DEOPT_IF func_version check described in the issue. Use the disassembly example as the reproducer and compare pyperformance results; done means these comprehension calls are statically avoided or no longer miss unnecessarily, with no regression in the reported benchmark.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.