apache / apache/arrow-java

[Java] Consume Substrait Plans

Open
#218 0 comments 0 reactions 0 assignees View on GitHub
Component: Documentation Type: enhancement
Dominant language
Java
Stars
94
Forks
152
Avg merge
3d 16h
Merged PRs (30d)
11

Description

### Describe the enhancement requested

The main purpose is to expose Java API to be able to Consume Substrait Plans as an input data.

**High level proposal will be:**

1. Create Java JNI Wrappers to call C++ Substrait API:

- Call `Result> ExecuteSerializedPlan` from JNI Wrappers
- Expose the result in a C-Data Interface objects contract

2. Create Java API to read C-Data Interface:
- Call Java JNI Wrapper to execute serialized Substrait Plan
- Convert C-Data Interface `ArrowArrayStream` into `ArrowReader`: `Data.importArrayStream(this.allocator, arrowArrayStream)`

Parent issue, see subtasks

- https://github.com/apache/arrow/issues/34223
- https://github.com/apache/arrow/issues/34224
- https://github.com/apache/arrow-java/issues/216
- https://github.com/apache/arrow/issues/34226

### Component(s)

Documentation, Java, Packaging

Contributor guide

Open the contributing guide

Research direction

Start by reading the proposed JNI entry point, ExecuteSerializedPlan, and the Java conversion call Data.importArrayStream(this.allocator, arrowArrayStream). Review the linked parent and subtasks to establish the scope. Done means Java can execute a serialized Substrait plan through JNI and expose the result as an ArrowReader using the C Data Interface.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, java
Domain
api, backend
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.