apache / apache/arrow-java

[Java] Represent a data element of a vector as a tree of ArrowBufPointer

Aperta
#275 1 commento 0 reazioni 0 assegnatari Vedi su GitHub
Type: enhancement
Lingua principale
Java
Stelle
94
Fork
152
Merge medio
3g 16h
PR unite (30g)
11

Descrizione

For a fixed/variable width vector, each of its data element can be represented as an ArrowBufPointer object, which represents a contiguous memory segment. This makes many tasks easier and more efficient (without memory copy): calculating hash code, comparing values, etc.

This cannot be achieved for complex vectors, because their values often reside in more than one contiguous memory regions. However, it can be seen that the contiguous memory regions for each data element forms a tree-like structure, whose leaf nodes are the contiguous memory regions. For example, a data element for a struct vector forms a tree, whose root corresponds to the struct vector, while the child vectors corresponds to the child nodes of the tree root.

In this issue, we provide a data structure that represents each data element of a vector as a tree, whose leaf nodes are ArrowBufPointers, representing contiguous memory regions for the data element.

With this data structure, many tasks also becomes easier and more efficient: calculating hash code, comparing vector elements (ordering & equality). In addition, we can do something that could not have been done in the past, like placing data elements into a hash table/hash set, etc.

**Reporter**: [Liya Fan](https://issues.apache.org/jira/browse/ARROW-7213) / @liyafan82
#### PRs and other links:
- [GitHub Pull Request apache/arrow#5883](https://github.com/apache/arrow/pull/5883)

**Note**: *This issue was originally created as [ARROW-7213](https://issues.apache.org/jira/browse/ARROW-7213). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia con la descrizione originale di ARROW-7213 e GitHub Pull Request #5883, che sono gli unici riferimenti all’implementazione dell’issue. Determina i tipi di vettori, le API e il comportamento trattati; il lavoro è completato quando esiste una rappresentazione ad albero utilizzabile per gli elementi vettoriali e supporta le operazioni successive indicate senza copie non necessarie.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
java
Ambito
data
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
25/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.