apache / apache/arrow-java

[Java] allocate new buffer code doesn't release extra allocated buffer properly

未关闭
#417 2 条评论 0 个 reaction 已指派 0 人 在 GitHub 查看
Type: bug
主要语言
Java
星标
94
派生
152
平均合并
3 天 16 小时
30 天内合并 PR
11

描述

[Class BaseValueVector](https://github.com/apache/arrow/blob/master/java/vector/src/main/java/org/apache/arrow/vector/BaseValueVector.java) 's method allocFixedDataAndValidityBufs on line#162 allocates a buffer in power of 2 size. After that, it has a code to release extra buffer. For that, it calculates the extra buffer from allocated size "bufferSize" but in my opinion, it should take original "valueCount" to find the extra buffer size. 

Here, I see a problem in line#162, where its taking "bufferSize" to find the extra allocated bytes. It should be "valueCount\*typeWidth + valueCount/8".

Here is an example for that. Let's take 1000 ints. Then,
valueCount = 1000 ints
typeWidth = 4 bytes
validitiyBufferSize = 125 bytes
valueBufferSize = 4000 bytes
combinedSize(valueBufferSize + validityBufferSize) = 4128 bytes (multiple of 8)
combinedSizeWith2ThePowerSize = 8192 bytes, this will be "bufferSize" at line#152.

With the above calculation, this code should release (combinedSizeWith2ThePowerSize - combinedSize) = 4064 bytes. But, this is not happening.

 

 

**Reporter**: [Hitesh Khamesra](https://issues.apache.org/jira/browse/ARROW-5057)
#### PRs and other links:
- [GitHub Pull Request apache/arrow#4079](https://github.com/apache/arrow/pull/4079)

**Note**: *This issue was originally created as [ARROW-5057](https://issues.apache.org/jira/browse/ARROW-5057). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

贡献指南

打开贡献指南

调研方向

从 java/vector/src/main/java/org/apache/arrow/vector/BaseValueVector.java 的 allocFixedDataAndValidityBufs 开始,重点查看第 162 行附近的分配和额外缓冲区释放。跟踪 1,000 个 int 示例的缓冲区大小计算,并验证已完成的更改是否释放了预期的额外分配;注意,PR apache/arrow#4079 已经与此 issue 关联。

由索引模型根据 Issue 内容生成。

评估

技术栈
java
领域
data, performance
Issue 类型
缺陷
难度
2/5
预计耗时
1-3 小时
活跃度
停滞
描述清晰度
描述清楚
新手友好度
20/100

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。