[C++] Provide mechanism for disabling MemoryPoolStats at either compile- or run-time
- Dominant language
- C++
- Stars
- 17.1k
- Forks
- 4.3k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 88
Description
### Describe the enhancement requested
This is a follow-on issue to https://github.com/apache/arrow/issues/40646.
[MemoryPoolStats](https://github.com/apache/arrow/blob/main/cpp/src/arrow/memory_pool.h#L38) is used by Arrow's memory pools to track statistics related to memory usage by having each pool's `Allocate`, `Reallocate`, and `Free` methods call in its MemoryPoolStats's `UpdateAllocatedBytes` method. The cost of this tracking can be measured by updating a benchmark such as `AllocateDeallocate` to be multi-threaded and comparing a run with `UpdateAllocatedBytes` as-is versus a run with it commented out.
With 32 threads this benchmark is 10x faster on x86 when `AllocateDeallocate` is commented out. See https://gist.github.com/amoeba/b95102829280dbe2b1f64e6c23a5f594. https://github.com/apache/arrow/pull/40647 also has much more performance information.
https://github.com/apache/arrow/pull/40647 improves the performance under microbenchmarks but the cost cannot be completely avoided. I think it's reasonable to assume that real-world applications can likely detect this cost and may wish to disable `MemoryPoolStats` either at compile- or run-time.
### Component(s)
C++
Contributor guide
Assessment
This issue has not been assessed yet.