apache / apache/arrow-java

[Java] Write list of maps with the vector API

Open
#132 11 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

### Describe the usage question you have. Please include as many useful details as possible.

I'd like to store list of maps. Not sure if I am doing it wrong or there is a bug in the implementation.
Here is my example:
```java
try (ListVector listVector = ListVector.empty("sourceVector", allocator)) {

UnionListWriter listWriter = listVector.getWriter();

listWriter.allocate();

/*
* [{"one" -> 1, "two" -> 2}, {"three" -> 3}, null, {"four" -> null, "five" -> 5}]
*/

listWriter.setPosition(0);
listWriter.startList();

listWriter.map().startMap(); // <-- exception is thrown from here
listWriter.map().startEntry();
listWriter.map().key().varChar().writeVarChar("one");
listWriter.map().key().integer().writeInt(1);
listWriter.map().endEntry();
listWriter.map().startEntry();
listWriter.map().key().varChar().writeVarChar("two");
listWriter.map().key().integer().writeInt(2);
listWriter.map().endEntry();
listWriter.map().endMap();

listWriter.map().startMap();
listWriter.map().startEntry();
listWriter.map().key().varChar().writeVarChar("three");
listWriter.map().key().integer().writeInt(3);
listWriter.map().endEntry();
listWriter.map().endMap();

listWriter.writeNull();

listWriter.map().startMap();
listWriter.map().startEntry();
listWriter.map().key().varChar().writeVarChar("four");
listWriter.map().key().integer().writeNull();
listWriter.map().endEntry();
listWriter.map().startEntry();
listWriter.map().key().varChar().writeVarChar("five");
listWriter.map().key().integer().writeInt(5);
listWriter.map().endEntry();
listWriter.map().endMap();

listWriter.endList();
```

When I execute this I'm getting the following exception:
```
java.lang.UnsupportedOperationException: Cannot get simple type for type MAP
at org.apache.arrow.vector/org.apache.arrow.vector.types.Types$MinorType.getType(Types.java:807)
at org.apache.arrow.vector/org.apache.arrow.vector.complex.impl.PromotableWriter.getWriter(PromotableWriter.java:275)
at org.apache.arrow.vector/org.apache.arrow.vector.complex.impl.AbstractPromotableFieldWriter.getWriter(AbstractPromotableFieldWriter.java:80)
at org.apache.arrow.vector/org.apache.arrow.vector.complex.impl.AbstractPromotableFieldWriter.startMap(AbstractPromotableFieldWriter.java:114)
at org.apache.arrow.vector/org.apache.arrow.vector.complex.impl.PromotableWriter.startMap(PromotableWriter.java:53)
```

### Component(s)

Java

Contributor guide

Open the contributing guide

Research direction

Reproduce the Java example using ListVector, UnionListWriter, and the map-writing calls, then inspect the PromotableWriter stack trace where startMap() fails. Determine whether the vector API should support this list-of-maps usage; done means the example writes the stated maps, null list entry, and null map value without the reported exception.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.