pybind / pybind/pybind11

[BUG]: Having trouble creating immutable type with `tp_flags |= Py_TPFLAGS_IMMUTABLETYPE`

Open
#4,548 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug enhancement help wanted
Dominant language
C++
Stars
18k
Forks
2.3k
Avg merge
5d 17h
Merged PRs (30d)
10

Description

Required prerequisites
What version (or hash if on master) of pybind11 are you using?

2.10.3

Problem description

I'm trying to bind a C++ class to Python using py::class_<CustomClass>. I have some internal knowledge about my custom class, which should be immutable. So I want to turn on Py_TPFLAGS_IMMUTABLETYPE (since Python 3.10) in my custom Python type object's tp_flags.

I tried to set the tp_flags using py::custom_type_setup:

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.",
                                py::custom_type_setup([](PyHeapTypeObject* heap_type) {
                                    auto* type = &heap_type->ht_type;
#ifdef Py_TPFLAGS_IMMUTABLETYPE
                                    type->tp_flags |= Py_TPFLAGS_IMMUTABLETYPE;
#endif
                                }));

However, the typeobject becomes frozen immediately in PyType_Ready after calling the custom_type_setup_callback:

https://github.com/pybind/pybind11/blob/3cc7e4258c15a6a19ba5e0b62a220b1a6196d4eb/include/pybind11/detail/class.h#L716-L735

then a TypeError will raise at line 734 when importing the built extension:

https://github.com/pybind/pybind11/blob/3cc7e4258c15a6a19ba5e0b62a220b1a6196d4eb/include/pybind11/detail/class.h#L733-L735

which is equivalent to:

ImmutableCustom.__module__: str = mod.__name__

it raises:

>>> import _C
TypeError: cannot set '__module__' attribute of immutable type '_C.ImmutableCustom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: initialization failed

Also, after PyType_Ready, I cannot do cls.def_property_readonly(...).def(...); since it is immutable.


I tried to modify the tp_flags after I had fully initialized the class definition:

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.");

reinterpret_cast<PyTypeObject*>(ImmutableCustomTypeObject.ptr())->tp_name = "mymod.ImmutableCustomTypeObject";
py::setattr(ImmutableCustomTypeObject, "__module__", py::str("mymod"));

ImmutableCustomTypeObject
    .def_property_readonly(...)
    .def_property_readonly(...)
    .def(...)
    .def(...);

#ifdef Py_TPFLAGS_IMMUTABLETYPE
reinterpret_cast<PyTypeObject*>(ImmutableCustomTypeObject.ptr())->tp_flags |=
    Py_TPFLAGS_IMMUTABLETYPE;
#endif

I'm not sure if this is allowed to flip tp_flags if the type is already called with PyType_Ready.

Reproducible example code

Initialize a bound class with tp_flags |= Py_TPFLAGS_IMMUTABLETYPE (Python 3.10+):

auto ImmutableCustomTypeObject =
    py::class_<ImmutableCustom>(mod,
                                "ImmutableCustom",
                                "Instances of this class are immutable.",
                                py::custom_type_setup([](PyHeapTypeObject* heap_type) {
                                    auto* type = &heap_type->ht_type;
#ifdef Py_TPFLAGS_IMMUTABLETYPE
                                    type->tp_flags |= Py_TPFLAGS_IMMUTABLETYPE;
#endif
                                }));

Then import the C++ extension in Python:

>>> import _C
TypeError: cannot set '__module__' attribute of immutable type '_C.ImmutableCustom'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ImportError: initialization failed
Is this a regression? Put the last known working version here if it is.

Not a regression

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in include/pybind11/detail/class.h around the custom_type_setup callback and the subsequent PyType_Ready and module assignment. Reproduce the failure with the provided immutable-type example, then trace when pybind11 finishes class initialization; done requires an agreed supported behavior or a documented limitation for Py_TPFLAGS_IMMUTABLETYPE.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
backend-api-design, tooling
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
32/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.