Add _DuringGC functions for tp_traverse
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Feature or enhancement
As seen in https://discuss.python.org/t/adding-c-api-for-use-in-tp-traverse/105331/13 and https://github.com/capi-workgroup/decisions/issues/95:
There are newly documented restrictions on tp_traverse:
The traversal function must not have any side effects. It must not modify the reference counts of any Python objects nor create or destroy any Python objects.
But, whether a C-API function modifies some reference count is an implementation detail that we can change at any time. Core devs can add logging, an audit event, or a warning, to virtually any function.
This implies that you can't safely use any C API function in tp_traverse, unless it guarantees that it has no side effects.
I propose to add such side-effect-free versions of some functions useful in tp_traverse, with a _DuringGC suffix -- namely:
PyObject_GetTypeData_DuringGCPyObject_GetItemData_DuringGCPyType_GetModuleState_DuringGCPyModule_GetState_DuringGCPyModule_GetToken_DuringGCPyType_GetBaseByToken_DuringGCPyType_GetModule_DuringGCPyType_GetModuleByToken_DuringGC
Compared to their non-suffixed variants, the main difference is that we promise that these will remain safe to use in tp_traverse. The other differences are that these functions cannot fail, and (where applicable) they return borrowed references rather than strong ones.
Also, Py_VISIT, Py_SIZE, PyObject_VisitManagedDict as well as the visitproc that Python passes to tp_traverse will be documented to work like this.
[edit] I'll also include a few ones useful in these: PyObject_TypeCheck, PyType_IsSubtype, PyType_HasFeature, Py{<type>}_Check, Py{<type>}_CheckExact.
Users that call any _DuringGC function (including some type's tp_traverse) must manually must ensure that the type, MRO, and layout of any argument doesn't concurrently change. That means only calling them from tp_traverse.
Linked PRs
- gh-145925
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the linked C-API discussion, decisions issue 95, and PR gh-145925. Review the listed _DuringGC functions and the related Py_VISIT, Py_SIZE, visitproc, and type-checking APIs. Done means the proposed side-effect-free APIs and their tp_traverse safety guarantees are implemented and documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100