apache / apache/arrow-java

[Java] Represent a data element of a vector as a tree of ArrowBufPointer

Aberta
#275 1 comentário 0 reações 0 responsáveis Ver no GitHub
Type: enhancement
Linguagem predominante
Java
Estrelas
94
Forks
152
Merge médio
3d 16h
PRs com merge (30d)
11

Descrição

For a fixed/variable width vector, each of its data element can be represented as an ArrowBufPointer object, which represents a contiguous memory segment. This makes many tasks easier and more efficient (without memory copy): calculating hash code, comparing values, etc.

This cannot be achieved for complex vectors, because their values often reside in more than one contiguous memory regions. However, it can be seen that the contiguous memory regions for each data element forms a tree-like structure, whose leaf nodes are the contiguous memory regions. For example, a data element for a struct vector forms a tree, whose root corresponds to the struct vector, while the child vectors corresponds to the child nodes of the tree root.

In this issue, we provide a data structure that represents each data element of a vector as a tree, whose leaf nodes are ArrowBufPointers, representing contiguous memory regions for the data element.

With this data structure, many tasks also becomes easier and more efficient: calculating hash code, comparing vector elements (ordering & equality). In addition, we can do something that could not have been done in the past, like placing data elements into a hash table/hash set, etc.

**Reporter**: [Liya Fan](https://issues.apache.org/jira/browse/ARROW-7213) / @liyafan82
#### PRs and other links:
- [GitHub Pull Request apache/arrow#5883](https://github.com/apache/arrow/pull/5883)

**Note**: *This issue was originally created as [ARROW-7213](https://issues.apache.org/jira/browse/ARROW-7213). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Guia de contribuição

Abrir o guia de contribuição

Direção de pesquisa

Comece pela descrição original de ARROW-7213 e pelo GitHub Pull Request #5883, que são as únicas referências de implementação da issue. Determine os tipos de vetor, as APIs e o comportamento abrangidos por eles; o trabalho estará concluído quando existir uma representação em árvore utilizável para elementos vetoriais e ela oferecer suporte às operações subsequentes indicadas sem cópia desnecessária.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
java
Domínio
data
Tipo de issue
Funcionalidade
Dificuldade
5/5
Tempo estimado
Mais de uma semana
Status de atividade
Estagnada
Clareza
Precisa de esclarecimento
Facilidade para iniciantes
25/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.