apache / apache/arrow-java

[Java] Add API for getBufferSizeFor() with density to BaseVariableWidthVector

Aberta
#389 0 comentários 0 reações 0 responsáveis Ver no GitHub
Type: enhancement
Linguagem predominante
Java
Estrelas
94
Forks
152
Merge médio
3d 16h
PRs com merge (30d)
11

Descrição

Following the discussion on https://github.com/apache/arrow/pull/9187.

Proposed API in BaseVariableWidthVector.java:

```java

/**
* Get the potential buffer size for a particular number of records and density.
* @param valueCount desired number of elements in the vector
* @param density average number of bytes per variable width element
* @return estimated size of underlying buffers if the vector holds
* a given number of elements
*/
public int getBufferSizeFor(final int valueCount, double density)
```

The current `getBufferSizeFor(int valueCount)` for BaseVariableWidthVector requires that validity and offset vectors have already been allocated for at least the given `valueCount`. If the aim of this method is to estimate memory usage for a value count, it's not very useful because it can only give sizes for less than or equal value counts in the currently allocated vector.

A better approach for approximating memory usage is to include a density argument, along with value count. Then the buffer estimate does not require the validity and offset vector to have any allocation. This also is inline with `setInitialCapacity(int valueCount, double density)`

NOTE: this API should also be added to BaseLargeVariableWidthVector and possibly BaseRepeatedValueVector(Large) as well.

**Reporter**: [Bryan Cutler](https://issues.apache.org/jira/browse/ARROW-11739) / @BryanCutler

**Note**: *This issue was originally created as [ARROW-11739](https://issues.apache.org/jira/browse/ARROW-11739). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Guia de contribuição

Abrir o guia de contribuição

Direção de pesquisa

Comece com BaseVariableWidthVector.java e compare o getBufferSizeFor(int valueCount) existente com setInitialCapacity(int valueCount, double density). Revise a discussão do pull request vinculado e, em seguida, verifique se a mesma API se aplica a BaseLargeVariableWidthVector e BaseRepeatedValueVector(Large); considera-se concluído quando a API baseada em densidade solicitada estiver definida de forma consistente em todas as classes de vetores aplicáveis.

Escrita pelo modelo de indexação a partir do texto da issue.

Avaliação

Stack de tecnologia
java
Domínio
backend-api-design, data
Tipo de issue
Funcionalidade
Dificuldade
4/5
Tempo estimado
3-5 dias
Status de atividade
Estagnada
Clareza
Razoavelmente clara
Facilidade para iniciantes
35/100

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.