[Python] Multiple warnings and asserts triggered in debug CPython 3.11
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 91
Description
### Describe the bug, including details regarding any error messages, version, and platform.
CPython [can be built in debug mode](https://docs.python.org/3/using/configure.html#python-debug-build) to catch some maybe-fatal-maybe-not errors. We have such python3.11 configured with `--with-pydebug` and pyarrow 10.0.1 installed, here is an example of gc warning
```
>>> import pyarrow as pa
>>> table = pa.table({'a': [1]})
gc:0: ResourceWarning: Object of type pyarrow.lib.Int64Array is not untracked before destruction
```
similar code sometimes triggers assertion
```
gc:0: ResourceWarning: Object of type pyarrow.lib.UInt16Array is not untracked before destruction
Modules/gcmodule.c:442: update_refs: Assertion "gc_get_refs(gc) != 0" failed
Enable tracemalloc to get the memory block allocation traceback
object address : 0x7f804e8762e0
object refcount : 0
object type : 0x7f80fcc3f5e0
object type name: pyarrow.lib.UInt16Array
object repr :
Fatal Python error: _PyObject_AssertFailed: _PyObject_AssertFailed
Python runtime state: initialized
```
Another crash
```
>>> import pyarrow as pa
>>> pa.table({0: []})
python: Objects/typeobject.c:1068: type_call: Assertion `!_PyErr_Occurred(tstate)' failed.
Aborted (core dumped)
```
### Component(s)
Python
Contributor guide
Research direction
Reproduce the warnings and crashes with a debug CPython 3.11 build, pyarrow 10.0.1, and the examples in the issue. Inspect the reported paths in Modules/gcmodule.c and Objects/typeobject.c, then trace the pyarrow objects involved; done means the examples no longer emit the warnings, assertions, or fatal errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend, data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100