apache / apache/arrow-adbc

about the bulkIngest feature

Open
#1,232 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
627
Forks
217
Avg merge
22h 22m
Merged PRs (30d)
64

Description

In `FlightInfoReader`, adbc's java client supports multiple endpoint reads (although in the java client, these endpoints are handled serially), which is great!

However, the bulkIngest function, when importing, can only import data to one endpoint.
Is it possible to support parallel import of multiple endpoints?

For example, we use an MPP type database, we need to do bulkIngest operations, the master returns multiple endpoints (corresponding to different workers), and then sends data to these endpoints at the same time.

Contributor guide

Open the contributing guide

Research direction

Start with FlightInfoReader and the Java client's bulkIngest path; compare its single-endpoint import behavior with the existing handling of multiple endpoints. Use the MPP example in the issue to define the expected behavior: data should be imported to multiple returned endpoints in parallel, with coverage for that outcome.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend-api-design, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.