Reading only a subset of columns
- Dominant language
- Julia
- Stars
- 312
- Forks
- 78
- PR merge metrics
- No merged PRs in 30d
Description
Please correct me if this is possible already. I looked through the source code and the documentation and did not find a clear way to do this: basically, I want to read a FeatherV2 file, but not mmap every single column. I already know which columns I need and I'd like to tell `Arrow.Table` the subset of columns I want read into memory.
This is similar to [this issue on Feather.jl](https://github.com/JuliaData/Feather.jl/issues/36).
This seems to be possible in the R arrow package using `col_select`.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the Arrow.Table entry point for reading FeatherV2 files and review how column selection and memory mapping are handled. Compare the requested behavior with Feather.jl issue 36 and R arrow's col_select option; done means callers can select columns without mapping every column into memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100