apache / apache/arrow-java

Unable to create a list of Maps with Decimal key or value

Aberta
#69 1 comentário 1 reação 1 responsável Reivindicada por @vibhatha Ver no GitHub
Type: usage
Linguagem predominante
Java
Estrelas
94
Forks
152
Merge médio
3d 16h
PRs com merge (30d)
11

Descrição

### Describe the usage question you have. Please include as many useful details as possible.

Tested on Arrow 15.0.2

I would like to create a list vector of maps. This works fine for most data types.

However when I'm creating a list of Map it fails with following error:

```
java.lang.UnsupportedOperationException: Cannot get simple type for type MAP

at org.apache.arrow.vector.types.Types$MinorType.getType(Types.java:807)
at org.apache.arrow.vector.complex.impl.PromotableWriter.getWriter(PromotableWriter.java:274)
at org.apache.arrow.vector.complex.impl.AbstractPromotableFieldWriter.getWriter(AbstractPromotableFieldWriter.java:83)
at org.apache.arrow.vector.complex.impl.AbstractPromotableFieldWriter.startMap(AbstractPromotableFieldWriter.java:117)
at org.apache.arrow.vector.complex.impl.PromotableWriter.startMap(PromotableWriter.java:52)
at io.trino.plugin.hive.functions.TestUnloadArrow.testListOfMap(TestUnloadArrow.java:966)
at java.base/java.lang.reflect.Method.invoke(Method.java:580)
at java.base/java.util.concurrent.RecursiveAction.exec(RecursiveAction.java:194)
at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:387)
at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1312)
at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1843)
at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1808)
at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:188)

```

```
@Test
public void testListOfMap() {
try (ListVector from = ListVector.empty("v", new RootAllocator())) {
UnionListWriter listWriter = from.getWriter();
listWriter.allocate();

// write null, [null,{"f1":1,"f2":2},null,
// {"f1":1,"f2":2},null] alternatively
for (int i = 0; i < 10; i++) {
listWriter.setPosition(i);
if (i % 2 == 0) {
listWriter.writeNull();
continue;
}
listWriter.startList();
listWriter.map().startMap();
listWriter.map().writeNull();
listWriter.map().startEntry();
listWriter.map().key();
listWriter.map().decimal().writeDecimal(BigDecimal.valueOf(2.0));
listWriter.map().value();
listWriter.map().decimal().writeDecimal(BigDecimal.valueOf(3.0));
listWriter.endEntry();
listWriter.map().endMap();
listWriter.endList();
}
from.setValueCount(10);

System.out.println(from);
}
}
```

Struct writing works fine, however the Decimal instantation requires scale and precision to be passed in.

Following test works fine.

```
@Test
public void testListOfStruct() {
try (ListVector from = ListVector.empty("v", new RootAllocator())) {

UnionListWriter listWriter = from.getWriter();
listWriter.allocate();

// write null, [null,{"f1":1,"f2":2},null,
// {"f1":1,"f2":2},null] alternatively
for (int i = 0; i < 10; i++) {
listWriter.setPosition(i);
if (i % 2 == 0) {
listWriter.writeNull();
continue;
}
listWriter.startList();
listWriter.struct().writeNull();
listWriter.struct().start();
listWriter.struct().decimal("f1", 1, 2).writeDecimal(BigDecimal.valueOf(2.0));
listWriter.struct().end();
listWriter.struct().writeNull();
listWriter.struct().start();
listWriter.struct().decimal("f1", 1, 2).writeDecimal(BigDecimal.valueOf(3.0));
listWriter.struct().end();
listWriter.struct().writeNull();
listWriter.endList();
}
from.setValueCount(10);

System.out.println(from);
}
}
```

Is there any way to work around this. I have tried to specify the FieldType when creating the ListVector but this fieldType has no child types only the parent type (LIST).

### Component(s)

Java

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.