apache / apache/arrow-java

[Java] Netty memory release

Open
#215 0 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

### Describe the bug, including details regarding any error messages, version, and platform.

Since when we use netty as my memory allocator manager, I found that direct memory is not released after vector is closed.
```
Field field = new Field("test", FieldType.nullable(new ArrowType.Utf8()), null);
RootAllocator allocator = new RootAllocator(Long.MAX_VALUE);
FieldVector vector = field.createVector(allocator);
for (int i = 0; i < 100000; i++){
((VarCharVector)vector).setSafe(i, "test".getBytes(StandardCharsets.UTF_8));
}

System.out.println("before close memory: " + usedDirectMemory() + " bytes"); //before close memory: 4194304 bytes
vector.close();
allocator.close();
System.out.println("after closed memory: " + usedDirectMemory() + " bytes"); //after closed memory: 4194304 bytes
```
I can address the problem that if the inner allocator `PooledByteBufAllocator` won't free the memory chunk , I am not sure if this is expected.

### Component(s)

Java

Contributor guide

Open the contributing guide

Research direction

Start with the supplied Java reproduction using RootAllocator, VarCharVector, and PooledByteBufAllocator, then trace allocator close behavior and Netty's direct-memory handling. Confirm whether the 4194304-byte remainder is an actual leak or expected pooling behavior, and document or test the resulting expected behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.