apache / apache/arrow-java

Managing ownership in VectorSchemaRoot#addVector, recent changes miss the main fault.

Open
#1,142 4 comments 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

A recent change in VectorSchemaRoot#addVector provided a simple improvement, but completely misses a major shortcoming of this routine and, almost certainly, others like it. Based on the interest in that issue (#301), this issue might be of interest to @axreldable and @jbonofre.

One cannot change the `FieldVector`s belonging to `VectorSchemaRoot`s by simply rearranging the elements of their `List`s without considering the ownership of their underlying buffers. This can cause various problems, particularly when `VectorSchemaRoot`s are closed. This is why `TransferPair` exists.

I would probably need some help with improving this as I'm quite new to using Apache Arrow (and very new to using GitHub). But I have written a possibly improved implementation if this would be of interest and would be open to collaborating on rectifying this bug.

Contributor guide

Open the contributing guide

Research direction

Start with VectorSchemaRoot#addVector and read the related discussion in issue #301. Investigate how rearranging FieldVectors affects ownership of their underlying buffers and how VectorSchemaRoots behave when closed. Done means the ownership transfer is handled correctly and closing roots no longer causes the described buffer problems.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.