apache / apache/arrow

[Python] set_memory_pool() is misleading

Open
#47,595 3 comments 0 reactions 0 assignees View on GitHub
Component: Parquet Component: Python Type: bug
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the bug, including details regarding any error messages, version, and platform.

The PyArrow `set_memory_pool()` function does not set the C++ default memory pool, but a separate PyArrow-specific default memory pool (which does not seem to be used everywhere, see https://github.com/apache/arrow/issues/47266#issuecomment-3207576731). So in many cases, calling it does not change the underlying allocator for later operations.

We could either fix this function to set the default C++ memory pool (but we would need to add an API for that), or deprecate it.

### Component(s)

Parquet, Python

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.