apache / apache/arrow-java

[Java] ComplexWriter should initialize nested writers when container vector is already populated

Open
#295 1 comment 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

It's possible and sometimes useful to use reuse a nested vector that was populated in a previous ComplexWriter. The new ComplexWriter should be aware of the fields that are present in the vector.

As it is right now, if a particular column were determined to be a specific type (or a union type), but the new writer finds a new type, the original type may be thrown out. What should happen is that the type should be promoted to union (or have a new subtype added to the union field).

**Reporter**: [Steven Phillips](https://issues.apache.org/jira/browse/ARROW-326) / @StevenMPhillips

**Note**: *This issue was originally created as [ARROW-326](https://issues.apache.org/jira/browse/ARROW-326). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by locating ComplexWriter and the code that initializes nested writers from an existing container vector. Reproduce the populated-vector case described in the issue, then verify that existing types are preserved and new types promote the field to a union or add a subtype without losing prior types.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.