[Python][C++] Simplify/Validate PyObject refcount in udf.cc
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the bug, including details regarding any error messages, version, and platform.
In udf.cc, there is interaction between C++ and Python where the C++ code holds a user defined Python object inside a Kernel and invoke it over input data.
Due to this interaction, there are a number of `Py_INCREF` call as well as using `OwnRef` to manage the refcount for the PyObject (which represents the user defined function. The current code for managing it is either over complicated and/or bug where the C++ code might be over increasing the refcount. While this is not a huge issue (it probably cause the Python function to never be garbage collected), it could use some clean up / documentation as least to improve maintainability.
Appendix:
* Related thread: https://github.com/apache/arrow/pull/35514#discussion_r1205081272
### Component(s)
C++
Contributor guide
Research direction
Start by reading udf.cc and the related discussion in apache/arrow#35514, tracing each Py_INCREF and OwnRef use around the user-defined function held by the Kernel. Done means the reference-count handling is simpler, documented where needed, and no longer over-increments the PyObject.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100