[Java] allocate new buffer code doesn't release extra allocated buffer properly
- Dominant language
- Java
- Stars
- 94
- Forks
- 152
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 11
Description
[Class BaseValueVector](https://github.com/apache/arrow/blob/master/java/vector/src/main/java/org/apache/arrow/vector/BaseValueVector.java) 's method allocFixedDataAndValidityBufs on line#162 allocates a buffer in power of 2 size. After that, it has a code to release extra buffer. For that, it calculates the extra buffer from allocated size "bufferSize" but in my opinion, it should take original "valueCount" to find the extra buffer size.
Here, I see a problem in line#162, where its taking "bufferSize" to find the extra allocated bytes. It should be "valueCount\*typeWidth + valueCount/8".
Here is an example for that. Let's take 1000 ints. Then,
valueCount = 1000 ints
typeWidth = 4 bytes
validitiyBufferSize = 125 bytes
valueBufferSize = 4000 bytes
combinedSize(valueBufferSize + validityBufferSize) = 4128 bytes (multiple of 8)
combinedSizeWith2ThePowerSize = 8192 bytes, this will be "bufferSize" at line#152.
With the above calculation, this code should release (combinedSizeWith2ThePowerSize - combinedSize) = 4064 bytes. But, this is not happening.
**Reporter**: [Hitesh Khamesra](https://issues.apache.org/jira/browse/ARROW-5057)
#### PRs and other links:
- [GitHub Pull Request apache/arrow#4079](https://github.com/apache/arrow/pull/4079)
**Note**: *This issue was originally created as [ARROW-5057](https://issues.apache.org/jira/browse/ARROW-5057). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start in java/vector/src/main/java/org/apache/arrow/vector/BaseValueVector.java at allocFixedDataAndValidityBufs, especially the allocation and extra-buffer release around line 162. Trace the buffer-size calculations for the 1,000-int example and verify that the completed change releases the expected extra allocation; note that PR apache/arrow#4079 is already linked to this issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data, performance
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 20/100