apache / apache/arrow-adbc

Rust & Adbc RecordBatchReader behviour

Open
#3,827 4 comments 0 reactions 0 assignees View on GitHub
Type: question
Dominant language
C#
Stars
627
Forks
217
Avg merge
17h
Merged PRs (30d)
57

Description

### What would you like help with?

Must a `RecordBatchReader` be considered closed after it returns one error? Or could a `RecordBatchReader` implementation choose to have recovery which allows for calling `next` after a previous `next` returned an error?

Alternatively, this question might be equivalent to asking if `next` returning `None` means "No more data" or "All data received successfully".

The only hint I found was the c definition of `get_next` in `adbc.h` that says "if no error and the array is released, the stream has ended" which means if there is one error the stream *has not* ended and never will.

Finally, if `next` returns an error and is invoked again and then returns `None` is that valid behavior when it means the stream ended *Unsuccessfully*?

thank you

Contributor guide

Open the contributing guide

Research direction

Start by comparing Rust RecordBatchReader's next semantics with the get_next definition in adbc.h, especially the wording about errors and released arrays. The work is done when the allowed post-error behavior and the meaning of None are decided and documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, rust
Domain
backend-api-design, databases
Issue type
Documentation
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.