NativeMemoryAllocator doesn't remove all memory pressure
- Dominant language
- C#
- Stars
- 39
- Forks
- 30
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 16
Description
### Describe the bug, including details regarding any error messages, version, and platform.
The `NativeMemoryAllocator` always over-allocates by the alignment amount to ensure the final buffer is aligned. When it [adds GC pressure](https://github.com/apache/arrow-dotnet/blob/710f6e47bd20c5a0dca471a27d67c2fff65d581c/src/Apache.Arrow/Memory/NativeMemoryAllocator.cs#L45) via `GC.AddMemoryPressure()`, this extra padding is accounted for. However, when `NativeAllocationOwner` [releases the memory allocation](https://github.com/apache/arrow-dotnet/blob/710f6e47bd20c5a0dca471a27d67c2fff65d581c/src/Apache.Arrow/Memory/NativeMemoryAllocator.cs#L58), it doesn't include the extra padding when it calls `GC.RemoveMemoryPressure()` (the `length` parameter is the size of the `Memory`, not the unmanaged allocation).
This doesn't appear to have long-term consequences, since the GC will periodically clear its unmanaged memory pressure counters, but my understanding is it can lead to extra garbage collections overall.
A tool like [`PerfView`](https://github.com/microsoft/perfview) can show events for each call to `AddMemoryPressure` and `RemoveMemoryPressure`. Running it with the following code shows a call to add memory pressure of 74 bytes and a call to remove memory pressure of 10 bytes:
```csharp
using Apache.Arrow.Memory;
MemoryAllocator.Default.Value.Allocate(10).Dispose();
```
Contributor guide
Assessment
This issue has not been assessed yet.