apache / apache/arrow-java

[Java] allocate new buffer code doesn't release extra allocated buffer properly

Open
#417 2 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

[Class BaseValueVector](https://github.com/apache/arrow/blob/master/java/vector/src/main/java/org/apache/arrow/vector/BaseValueVector.java) 's method allocFixedDataAndValidityBufs on line#162 allocates a buffer in power of 2 size. After that, it has a code to release extra buffer. For that, it calculates the extra buffer from allocated size "bufferSize" but in my opinion, it should take original "valueCount" to find the extra buffer size. 

Here, I see a problem in line#162, where its taking "bufferSize" to find the extra allocated bytes. It should be "valueCount\*typeWidth + valueCount/8".

Here is an example for that. Let's take 1000 ints. Then,
valueCount = 1000 ints
typeWidth = 4 bytes
validitiyBufferSize = 125 bytes
valueBufferSize = 4000 bytes
combinedSize(valueBufferSize + validityBufferSize) = 4128 bytes (multiple of 8)
combinedSizeWith2ThePowerSize = 8192 bytes, this will be "bufferSize" at line#152.

With the above calculation, this code should release (combinedSizeWith2ThePowerSize - combinedSize) = 4064 bytes. But, this is not happening.

 

 

**Reporter**: [Hitesh Khamesra](https://issues.apache.org/jira/browse/ARROW-5057)
#### PRs and other links:
- [GitHub Pull Request apache/arrow#4079](https://github.com/apache/arrow/pull/4079)

**Note**: *This issue was originally created as [ARROW-5057](https://issues.apache.org/jira/browse/ARROW-5057). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start in java/vector/src/main/java/org/apache/arrow/vector/BaseValueVector.java at allocFixedDataAndValidityBufs, especially the allocation and extra-buffer release around line 162. Trace the buffer-size calculations for the 1,000-int example and verify that the completed change releases the expected extra allocation; note that PR apache/arrow#4079 is already linked to this issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data, performance
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.