apache / apache/arrow-java

VectorAppender fails with OversizedAllocationException when appending to empty UnionVector

Abierto
#812 0 comentarios 3 reacciones 0 asignados Ver en GitHub
Type: bug
Lenguaje dominante
Java
Estrellas
94
Forks
152
Merge medio
3 d 16 h
PR fusionados (30 d)
11

Descripción

### Describe the bug, including details regarding any error messages, version, and platform.

**Description:**
When appending a `UnionVector` to an empty target `UnionVector`, [this](https://github.com/apache/arrow-java/blob/main/vector/src/main/java/org/apache/arrow/vector/util/VectorAppender.java#L556) while loop runs infinitely until the exception occurs:
```
org.apache.arrow.vector.util.OversizedAllocationException: Unable to expand the buffer
```
It seems to be a issue with the `UnionVector.getValueCapacity()` method. Logging the capacity before and after calling `reAlloc()` shows that it's always `0`.
Debugging further, it seems like the value capacity of the `internalStruct` in the `UnionVector` always returns `0` because it has no children so [size](https://github.com/apache/arrow-java/blob/main/vector/src/main/java/org/apache/arrow/vector/complex/NonNullableStructVector.java#L373) is `0`.
```
@Override
public int getValueCapacity() {
return Math.min(getTypeBufferValueCapacity(), internalStruct.getValueCapacity());
}
```

**Steps to reproduce**
```
@Test
public void testAppendEmptyTargetUnionVector() {
final int length = 5;

try (final UnionVector target = UnionVector.empty("target", allocator);
final UnionVector delta = UnionVector.empty("delta", allocator)) {

// populate the delta vector
delta.setType(0, Types.MinorType.FLOAT4);
delta.setType(1, Types.MinorType.FLOAT4);
delta.setType(2, Types.MinorType.FLOAT4);
delta.setType(3, Types.MinorType.FLOAT4);
delta.setType(4, Types.MinorType.FLOAT4);

Float4Vector deltaFloatVector = delta.getFloat4Vector();
deltaFloatVector.allocateNew();
ValueVectorDataPopulator.setVector(deltaFloatVector, 1f, 2f, 3f, 4f, 5f);
assertEquals(length, deltaFloatVector.getValueCount());
delta.setValueCount(length);

VectorAppender appender = new VectorAppender(target);
delta.accept(appender, null);

assertEquals(length, target.getValueCount());

for (int i = 0; i < length; i++) {
Object floatObj = target.getObject(i);
assertTrue(floatObj instanceof Float);
assertEquals(i+1, ((Float) floatObj).intValue());
}
}
}
```
This issue does not occur if any value is set in the target vector before appending.

Guía de contribución

Abrir la guía de contribución

Línea de trabajo

Comienza con el bucle de VectorAppender.java alrededor de la línea 556 e inspecciona UnionVector.getValueCapacity(), incluido NonNullableStructVector.java alrededor de la línea 373. Ejecuta la reproducción proporcionada testAppendEmptyTargetUnionVector y compara la capacidad antes y después de reAlloc(). Se considera terminado cuando añadir a un UnionVector vacío se completa sin OversizedAllocationException y conserva los cinco valores float.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
java
Área
data
Tipo de issue
Error
Dificultad
3/5
Tiempo estimado
1-2 días
Estado de actividad
Estancado
Claridad
Bien especificado
Aptitud para principiantes
55/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.