apache / apache/arrow-rs

[FlightSQL] Support `execute_ingest` for prepared statement

Open
#6,561 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
3.6k
Forks
1.3k
Avg merge
2d 18h
Merged PRs (30d)
169

Description

**Is your feature request related to a problem or challenge? Please describe what you are trying to do.**

We have a use case where we read data from a source, construct a record batch stream, and write the data using INSERT SQL + prepared statement. However, the current prepared statement only offers `set_parameters` + `execute_update`, which limits us to inserting one record batch at a time.

Since Arrow Flight SQL client supports `execute_ingest`, I suggest that the prepared statement provides a similar interface. This would allow the client to insert multiple record batches in one call, e.g., via `prepared_stmt.execute_ingest(stream)`.

**Describe the solution you'd like**

Implement a method `execute_ingest` for Arrow FlightSQL client that receives a stream of parameter bindings, sends the stream to the server, and waits for a stream of DoPut results.

**Describe alternatives you've considered**

**Additional context**

Contributor guide

Open the contributing guide

Research direction

Start with the Arrow Flight SQL prepared-statement client APIs, especially set_parameters and execute_update, then compare them with the existing execute_ingest flow. Done means a prepared statement accepts a stream of parameter bindings, sends it to the server, and returns the stream of DoPut results.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.