apache / apache/arrow-java

VectorAppender fails with OversizedAllocationException when appending to empty UnionVector

Ouverte
#812 0 commentaires 3 réactions 0 personnes assignées Voir sur GitHub
Type: bug
Langage dominant
Java
Étoiles
94
Forks
152
Merge moyen
3 j 16 h
PR mergées (30 j)
11

Description

### Describe the bug, including details regarding any error messages, version, and platform.

**Description:**
When appending a `UnionVector` to an empty target `UnionVector`, [this](https://github.com/apache/arrow-java/blob/main/vector/src/main/java/org/apache/arrow/vector/util/VectorAppender.java#L556) while loop runs infinitely until the exception occurs:
```
org.apache.arrow.vector.util.OversizedAllocationException: Unable to expand the buffer
```
It seems to be a issue with the `UnionVector.getValueCapacity()` method. Logging the capacity before and after calling `reAlloc()` shows that it's always `0`.
Debugging further, it seems like the value capacity of the `internalStruct` in the `UnionVector` always returns `0` because it has no children so [size](https://github.com/apache/arrow-java/blob/main/vector/src/main/java/org/apache/arrow/vector/complex/NonNullableStructVector.java#L373) is `0`.
```
@Override
public int getValueCapacity() {
return Math.min(getTypeBufferValueCapacity(), internalStruct.getValueCapacity());
}
```

**Steps to reproduce**
```
@Test
public void testAppendEmptyTargetUnionVector() {
final int length = 5;

try (final UnionVector target = UnionVector.empty("target", allocator);
final UnionVector delta = UnionVector.empty("delta", allocator)) {

// populate the delta vector
delta.setType(0, Types.MinorType.FLOAT4);
delta.setType(1, Types.MinorType.FLOAT4);
delta.setType(2, Types.MinorType.FLOAT4);
delta.setType(3, Types.MinorType.FLOAT4);
delta.setType(4, Types.MinorType.FLOAT4);

Float4Vector deltaFloatVector = delta.getFloat4Vector();
deltaFloatVector.allocateNew();
ValueVectorDataPopulator.setVector(deltaFloatVector, 1f, 2f, 3f, 4f, 5f);
assertEquals(length, deltaFloatVector.getValueCount());
delta.setValueCount(length);

VectorAppender appender = new VectorAppender(target);
delta.accept(appender, null);

assertEquals(length, target.getValueCount());

for (int i = 0; i < length; i++) {
Object floatObj = target.getObject(i);
assertTrue(floatObj instanceof Float);
assertEquals(i+1, ((Float) floatObj).intValue());
}
}
}
```
This issue does not occur if any value is set in the target vector before appending.

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez par la boucle de VectorAppender.java autour de la ligne 556 et inspectez UnionVector.getValueCapacity(), notamment NonNullableStructVector.java autour de la ligne 373. Exécutez la reproduction fournie testAppendEmptyTargetUnionVector et comparez la capacité avant et après reAlloc(). C’est terminé lorsque l’ajout à un UnionVector vide s’effectue sans OversizedAllocationException et préserve les cinq valeurs float.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
java
Domaine
data
Type d'issue
Bug
Difficulté
3/5
Temps estimé
1-2 jours
Activité
À l'abandon
Clarté
Clairement spécifiée
Accessibilité débutants
55/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.