apache / apache/arrow-julia

[feature request] support run-end encoded layout

Open
#476 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Julia
Stars
312
Forks
78
PR merge metrics
No merged PRs in 30d

Description

The Arrow spec has a "run-end encoded layout" (https://arrow.apache.org/docs/format/Columnar.html#run-end-encoded-layout) but it doesn't appear to be supported in Arrow.jl. Seems fairly straightforward to implement since the `run_ends` and `values` arrays are "child arrays" and not buffers, so we could "just" re-use teh existing functionality to represent the values and run-ends.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Arrow specification's run-end encoded layout section and inspect Arrow.jl's existing functionality for child arrays. The work is done when the run_ends and values child arrays can represent the run-end encoded layout in Arrow.jl.

Written by the indexing model from the issue text.

Assessment

Tech stack
julia
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.