apache / apache/arrow

[C++] Add a way to programatically set the default memory pool

Open
#48,135 4 comments 1 reaction 0 assignees View on GitHub
Component: C++ Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

Arrow currently allows setting the default memory pool only via the `ARROW_DEFAULT_MEMORY_POOL` environment variable. It would be useful for applications to set the default memory pool programmatically, especially when they provide custom memory pools that can not be selected via the environment variable or know that a specific implementation is optimal for their use case.

Two possible implementation approaches:

1. Mark default_memory_pool() as a weak symbol

Annotating default_memory_pool() with [[gnu::weak]] would let client code override the default memory pool at link/load time.
Pros: avoids static initialization ordering issues and avoids internal thread-synchronization requirements.

2. Provide a setter API

Introduce a function such as SetDefaultMemoryPool(MemoryPool*) for applications to configure the default memory pool directly.
Pros: explicit and easy to understand; could allow changes during runtime if desired.

I'd like to Arrow to provide a way to override the default memory pool, and I'm willing to contribute an implementation, but I'd like to get some feedback on the direction from the maintainers first.

### Component(s)

C++

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.