python / python/cpython

Reduce overhead of PyErr_CheckSignals

Open
#157,746 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core performance type-feature
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Feature or enhancement

Proposal:

PyErr_CheckSignals() is the API long-running C loops use to stay interruptible. In the common case (no signal arrived) the call currently does, in order:

  1. _PyRunRemoteDebugger() (since gh-131591, 3.14): a call that fetches the interpreter config via _PyInterpreterState_GetConfig() before looking at the pending flag;
  2. _Py_ThreadCanHandleSignals(): _Py_IsMainThread() calls PyThread_get_thread_ident() and therefore pthread_self(), plus _Py_IsMainInterpreter();
  3. only then _PyErr_CheckSignalsTstate(), whose first line is the cheap is_tripped test that returns immediately.

That is about 100 instructions per call.

Proposal

Test the cheap flags first, without changing what is done when they are set:

  • _PyErr_CheckSignalsTstate() does the main-thread check itself, after its existing is_tripped test, so PyErr_CheckSignals() and the eval loop just call it; the handler-running body moves to an out-of-line helper so the early return stays cheap;
  • _PyRunRemoteDebugger() tests debugger_pending_call before fetching the config.
Results

Non-PGO build, pyperf:

Benchmark main branch
str(12345) 68.6 ns 56.8 ns: 1.21x faster
repr(list(range(100))) 3.06 us 2.36 us: 1.29x faster
', '.join(map(str, range(100))) 6.90 us 5.67 us: 1.22x faster
'%s=%r' % (k, v) 192 ns 177 ns: 1.08x faster
print(*range(100), file=f) 15.6 us 13.5 us: 1.16x faster
csv.writer: 100 rows of 10 ints 72.5 us 60.1 us: 1.21x faster
repr(dataclass with 5 fields) 805 ns 720 ns: 1.12x faster
a * b (10 x 10 digits) 159 ns 102 ns: 1.56x faster
a // c (10 // 2 digits) 151 ns 121 ns: 1.24x faster

Notes: the gain for the arithmetic cases are not too important, they can be handled directly as well. See https://github.com/python/cpython/issues/157742.

Has this already been discussed elsewhere?

Related: gh-131591, gh-133465

Has this already been discussed elsewhere?

No response given

Links to previous discussion of this feature:

No response

Linked PRs
  • gh-157748

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading PyErr_CheckSignals(), _PyErr_CheckSignalsTstate(), and _PyRunRemoteDebugger(), focusing on the stated ordering of cheap flag checks and signal handling. Compare the proposed behavior and benchmarks with linked PR gh-157748; the issue is done when the no-signal path is cheaper without changing behavior when flags are set.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
performance
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.