Use `PyObject_GetTypeData` without the GIL/attached thread state
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Feature or enhancement
Proposal:
In Cython we've historically let people do things like:
cdef class C:
cdef int non_object_attribute
def f(C c):
with nogil:
value = c.non_object_attribute
...
i.e. we've allowed them to get direct access into extension types without the GIL (providing they aren't manipulating Python objects).
When using PEP 697 lookup this becomes a little trickier because access to the attribute has to go through PyObject_GetTypeData. Looking at the implementation of PyObject_GetTypeData, it doesn't appear to need the GIL (except in debug mode where it sanity-checks the type).
I think I can work around this by calculating the offset of the struct myself once at startup (using PyObject_GetTypeData) and then doing all the lookups manually via the offset. It also looks like nanobind use a similar trick in places so I wouldn't be alone in this abuse of the system.
However, it'd be nice not to have to do this and if PyObject_GetTypeData could somehow be exempted from the "C API needs the GIL" rule.
I do understand that people would probably prefer not to do this, so I'm happy to look at other options.
Has this already been discussed elsewhere?
I have already discussed this feature proposal on Discourse
Links to previous discussion of this feature:
I asked about this in https://discuss.python.org/t/does-pyobject-gettypedata-need-the-gil/104637/4 - it only generated a small amount of discussion.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading the PyObject_GetTypeData implementation and the linked Discourse discussion about whether it needs the GIL. Determine whether a supported GIL-free approach is possible for PEP 697 attribute access, and document a clear outcome or alternative for Cython users.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c, python
- Domain
- api
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 32/100