[Java] Improving Arrow Vector Reading performance
- Dominant language
- Java
- Stars
- 94
- Forks
- 152
- Avg merge
- 3d 16h
- Merged PRs (30d)
- 11
Description
Currently the read interface of Java Arrow Vector is quite slow because the access operation has to go through validity bit check and boundary check before it can actually load the data. Such a safety check incurs unnecessary overhead in actually computation.
The Arrow Vector and ArrowBuf should expose unsafe methods for advanced users to directly access underlying data without null check and boundary check.
**Reporter**: [Yurui Zhou](https://issues.apache.org/jira/browse/ARROW-5197) / @yuruiz
#### Subtasks:
- [ ] [[Java] Add hasNull flag to Vectors](https://github.com/apache/arrow-java/issues/415)
- [X] [[Java] Add unsafe access method to ArrowBuf](https://github.com/apache/arrow/issues/21674)
#### Related issues:
- [[Java] Add accessor methods for data buffers that skip null checking](https://github.com/apache/arrow/issues/15627) (relates to)
- [[Java] Provide light-weight arrow APIs](https://github.com/apache/arrow/issues/21675) (is related to)
#### PRs and other links:
- [GitHub Pull Request apache/arrow#4199](https://github.com/apache/arrow/pull/4199)
**Note**: *This issue was originally created as [ARROW-5197](https://issues.apache.org/jira/browse/ARROW-5197). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*
Contributor guide
Research direction
Start by reviewing the Java Arrow Vector and ArrowBuf access interfaces, then inspect the completed ArrowBuf work and PR apache/arrow#4199. The issue is done when unsafe access methods support direct reads without null and boundary checks, with the related vector work and performance behavior addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100