[Java] reduce heap footprint of ValueVectors
- 主要語言
- Java
- 星號
- 94
- 分支
- 152
- 平均合併
- 3 天 16 小時
- 30 天內合併 PR
- 11
描述
In some scenarios, we hold lots of value vectors in memory eg. during join, aggregation. The heap analysis shows that the costs are as follows for a simple IntVector (used VisualVM on mac) :
IntVector : 80 bytes
vector.types.pojo.FieldType : 41 bytes
vector.types.pojo.Field : 40 bytes
IntReaderImpl : 28 bytes
I'll use this Jira to track ways to reduce the heap usage.
**Reporter**: [Pindikura Ravindra](https://issues.apache.org/jira/browse/ARROW-5482) / @pravindra
#### Related issues:
- [[Java] remove FieldReader from ValueVector](https://github.com/apache/arrow-java/issues/411) (is a parent of)
- [[Java] add ValueVector constructors that take a Field object](https://github.com/apache/arrow/issues/21932) (is a parent of)
**Note**: *This issue was originally created as [ARROW-5482](https://issues.apache.org/jira/browse/ARROW-5482). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
貢獻指南
研究方向
先閱讀 Java ValueVector 實作以及相關的「remove FieldReader from ValueVector」issue (#411),然後使用 VisualVM 重現列出的 IntVector 堆積設定檔。完成標準是:測得報告中的 ValueVector 記憶體成本有所降低,並記錄受影響的情境和取捨。
由索引模型根據 Issue 內容生成。
評估
- 技術堆疊
- java
- 領域
- performance
- Issue 類型
- 重構
- 難度
- 5/5
- 預估耗時
- 一週以上
- 活躍度
- 停滯
- 描述清晰度
- 需要釐清
- 新手友好度
- 25/100