python / python/cpython

It isn't possible to temporarily resurrect objects safely in the Py3.15t limited API

Open
#152,406 8 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

interpreter-core topic-C-API type-bug
Dominant language
Python
Stars
77.2k
Forks
35.9k
PR merge metrics
PR metrics pending

Description

Bug report

Bug description:

In Cython we do something like

Py_SET_REFCNT(o, Py_REFCNT(o) + 1);
// Run user implemented deallocation code
Py_SET_REFCNT(o, Py_REFCNT(o) - 1);

The reason for this is that we don't want any refcounting (e.g. of temps) within the user code to accidentally trigger a recursive call to dealloc (the user code really shouldn't resurrect the object and it's their own problem if they do, but it is reasonable for it to generate temporaries, which may well be out of their control anyway).

This turns out not to be thread-safe in the freethreaded build because it makes TryIncRef succeed. Outside the limited API it's possible to fix by manual access to the reference count, but obviously not inside the limited API.

It would be nice to have a way of to avoid this.

  • Most convenient would be to properly access _PyObject_ResurrectStart and _PyObject_ResurrectEnd.
  • The other thing that could work is some way to defer the deallocation until the next GC cycle (when I think any TryIncRefs would have fall out of scope anyway). I suspect I could manually arrange to do that, although it feels fragile, possibly relies on the GC being single-threaded, and which I haven't tested (so may not work...)
CPython versions tested on:

3.15

Operating systems tested on:

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the limited-API surface involved in refcount resurrection and the free-threaded TryIncRef behavior described here, then compare it with _PyObject_ResurrectStart and _PyObject_ResurrectEnd and Cython issue #7769. Done means a defined, thread-safe limited-API mechanism that avoids unsafe temporary resurrection during user deallocation code.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, python
Domain
api
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.