apache / apache/arrow-java

[Java] FieldVector getFieldBuffers API should not set reader/writer indices

Open
#270 1 comment 0 reactions 0 assignees View on GitHub
Type: bug
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

Per discussion .

The fact that we have reader/writer settings in `getFieldBuffers` is wrong. To clarify, `getFieldBuffers` is distinct from `getBuffers`. The former should be for getting access to underlying data for higher-performance algorithms. The latter is for sending the data over the wire. Seems we've mixed up use of both.

 

Currently in `VectorUnloader`, we used `getFieldBuffers` to create `ArrowRecordBatch` that’s why we keep writer/reader indices in `getFieldBuffers`, we should use `getBuffers` instead.

**Reporter**: [Ji Liu](https://issues.apache.org/jira/browse/ARROW-7539) / @tianchen92
#### PRs and other links:
- [GitHub Pull Request apache/arrow#6156](https://github.com/apache/arrow/pull/6156)

**Note**: *This issue was originally created as [ARROW-7539](https://issues.apache.org/jira/browse/ARROW-7539). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start with VectorUnloader and compare its use of getFieldBuffers and getBuffers when creating ArrowRecordBatch instances. Read the FieldVector API discussion linked in the issue, then verify that getFieldBuffers provides underlying data without changing reader/writer indices while record-batch serialization uses getBuffers.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.