apache / apache/arrow-julia

Writing and Reading Random Access Files

Open
#434 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

Maybe related to #353

It is already possible to use Tables.partitioner to write record batches to a single Arrow file. However, when I read that file with Arrow.Table I do not know how to access a specific record batch like here: https://arrow.apache.org/docs/java/ipc.html#writing-and-reading-random-access-files

According to the docs, this should be possible but I am not sure if that is not implemented yet or simply not documented.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Tables.partitioner write path and Arrow.Table reading behavior, then compare them with the Apache Arrow Java random-access-files documentation linked in the issue. Check related issue #353 and determine whether record-batch access is implemented or only undocumented; done means the supported behavior and required user-facing guidance are clear.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.