VectorAppender fails with OversizedAllocationException when appending to empty UnionVector
- Langage dominant
- Java
- Étoiles
- 94
- Forks
- 152
- Merge moyen
- 3 j 16 h
- PR mergées (30 j)
- 11
Description
### Describe the bug, including details regarding any error messages, version, and platform.
**Description:**
When appending a `UnionVector` to an empty target `UnionVector`, [this](https://github.com/apache/arrow-java/blob/main/vector/src/main/java/org/apache/arrow/vector/util/VectorAppender.java#L556) while loop runs infinitely until the exception occurs:
```
org.apache.arrow.vector.util.OversizedAllocationException: Unable to expand the buffer
```
It seems to be a issue with the `UnionVector.getValueCapacity()` method. Logging the capacity before and after calling `reAlloc()` shows that it's always `0`.
Debugging further, it seems like the value capacity of the `internalStruct` in the `UnionVector` always returns `0` because it has no children so [size](https://github.com/apache/arrow-java/blob/main/vector/src/main/java/org/apache/arrow/vector/complex/NonNullableStructVector.java#L373) is `0`.
```
@Override
public int getValueCapacity() {
return Math.min(getTypeBufferValueCapacity(), internalStruct.getValueCapacity());
}
```
**Steps to reproduce**
```
@Test
public void testAppendEmptyTargetUnionVector() {
final int length = 5;
try (final UnionVector target = UnionVector.empty("target", allocator);
final UnionVector delta = UnionVector.empty("delta", allocator)) {
// populate the delta vector
delta.setType(0, Types.MinorType.FLOAT4);
delta.setType(1, Types.MinorType.FLOAT4);
delta.setType(2, Types.MinorType.FLOAT4);
delta.setType(3, Types.MinorType.FLOAT4);
delta.setType(4, Types.MinorType.FLOAT4);
Float4Vector deltaFloatVector = delta.getFloat4Vector();
deltaFloatVector.allocateNew();
ValueVectorDataPopulator.setVector(deltaFloatVector, 1f, 2f, 3f, 4f, 5f);
assertEquals(length, deltaFloatVector.getValueCount());
delta.setValueCount(length);
VectorAppender appender = new VectorAppender(target);
delta.accept(appender, null);
assertEquals(length, target.getValueCount());
for (int i = 0; i < length; i++) {
Object floatObj = target.getObject(i);
assertTrue(floatObj instanceof Float);
assertEquals(i+1, ((Float) floatObj).intValue());
}
}
}
```
This issue does not occur if any value is set in the target vector before appending.
Guide de contribution
Ouvrir le guide de contribution
Piste de recherche
Commencez par la boucle de VectorAppender.java autour de la ligne 556 et inspectez UnionVector.getValueCapacity(), notamment NonNullableStructVector.java autour de la ligne 373. Exécutez la reproduction fournie testAppendEmptyTargetUnionVector et comparez la capacité avant et après reAlloc(). C’est terminé lorsque l’ajout à un UnionVector vide s’effectue sans OversizedAllocationException et préserve les cinq valeurs float.
Rédigé par le modèle d'indexation à partir du texte de l'issue.
Évaluation
- Stack technique
- java
- Domaine
- data
- Type d'issue
- Bug
- Difficulté
- 3/5
- Temps estimé
- 1-2 jours
- Activité
- À l'abandon
- Clarté
- Clairement spécifiée
- Accessibilité débutants
- 55/100