apache / apache/arrow-julia

explanation of Arrow.Stream vs. Arrow.Table seems ambiguous

Open
#472 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

https://github.com/apache/arrow-julia/blob/f8d2203b07380e1423723b5bfe32356aa1239284/docs/src/manual.md?plain=1#L189

Italicizing *iterate* for the second instance vs the first doesn't seem to illustrate in any unambiguous difference.

Does "Table" iterate batch units, while "Stream" iterates record units?

As I understand, a stream might be more like "on-demand" iteration. -like a generator, which only advances when necessary or able (space opens in a fixed buffer), while "Table" might "eagerly" load an entire table (batch), at a time.

It's not clear to me what are the units of each iteration, and the difference if both are iterating a table.

Maybe it would be simpler to see, if I just look at the code. :-)

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the linked passage around line 189 of docs/src/manual.md, then inspect the repository's Arrow.Stream and Arrow.Table usage to determine what each iteration yields and how consumption differs. Update the explanation so the iteration units and on-demand or eager behavior are unambiguous, and verify the revised wording against the documented API.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
documentation
Issue type
Documentation
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.