apache / apache/arrow

[Python] SafeCallIntoPython should error out if interpreter is finalizing/is not initialized

Open
#46,945 1 comment 0 reactions 0 assignees View on GitHub
Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

`SafeCallIntoPython` is used to construct a callback that can call into Python at arbitrary times from C++.

In some cases, however, it is too late to be able to execute Python code, or even take the GIL, because the Python interpreter has been finalized. This can happen if some async tasks are still running after the user stopped the operation. An example can be found [here](https://github.com/apache/arrow/issues/43497#issuecomment-3018933031).

This will not be completely error-proof, that will have to wait for [PEP 788](https://peps.python.org/pep-0788/), which is now accepted, to be implemented... hopefully in 3.15.

### Component(s)

Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.