apache / apache/arrow-java

[Java] Represent a data element of a vector as a tree of ArrowBufPointer

未關閉
#275 1 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
Type: enhancement
主要語言
Java
星號
94
分支
152
平均合併
3 天 16 小時
30 天內合併 PR
11

描述

For a fixed/variable width vector, each of its data element can be represented as an ArrowBufPointer object, which represents a contiguous memory segment. This makes many tasks easier and more efficient (without memory copy): calculating hash code, comparing values, etc.

This cannot be achieved for complex vectors, because their values often reside in more than one contiguous memory regions. However, it can be seen that the contiguous memory regions for each data element forms a tree-like structure, whose leaf nodes are the contiguous memory regions. For example, a data element for a struct vector forms a tree, whose root corresponds to the struct vector, while the child vectors corresponds to the child nodes of the tree root.

In this issue, we provide a data structure that represents each data element of a vector as a tree, whose leaf nodes are ArrowBufPointers, representing contiguous memory regions for the data element.

With this data structure, many tasks also becomes easier and more efficient: calculating hash code, comparing vector elements (ordering & equality). In addition, we can do something that could not have been done in the past, like placing data elements into a hash table/hash set, etc.

**Reporter**: [Liya Fan](https://issues.apache.org/jira/browse/ARROW-7213) / @liyafan82
#### PRs and other links:
- [GitHub Pull Request apache/arrow#5883](https://github.com/apache/arrow/pull/5883)

**Note**: *This issue was originally created as [ARROW-7213](https://issues.apache.org/jira/browse/ARROW-7213). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

貢獻指南

開啟貢獻指南

研究方向

從 ARROW-7213 的原始描述和 GitHub Pull Request #5883 開始,它們是該 issue 唯一的實作參考。確定其中涵蓋的向量型別、API 和行為;當存在可用於向量元素的樹狀表示,並且能夠在不進行不必要複製的情況下支援其中所述的後續操作時,即視為完成。

由索引模型根據 Issue 內容生成。

評估

技術堆疊
java
領域
data
Issue 類型
功能
難度
5/5
預估耗時
一週以上
活躍度
停滯
描述清晰度
需要釐清
新手友好度
25/100

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。