apache / apache/arrow

[Docs][C++] Guide to to work with RLE data (in kernels?)

Open
#32,340 0 comments 0 reactions 0 assignees View on GitHub
Component: C++ Component: Documentation Type: task
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

Topics I can think of right now:
- introducing some utility functions for working with RLE data that I'm collecting currently as rle_util.h on my filter branch
- how physical/logical indices work, especially in combination with arrow offsets
- how common kinds of kernels can be converted to RLE. E.g scalar kernels with a single input could automatically be run on the internal data array of RLE. Scalar with multiple inputs and some kinds of vector algorithms should use VisitMergedRuns...

**Reporter**: [Tobias Zagorni](https://issues.apache.org/jira/browse/ARROW-17029) / @zagto

**Note**: *This issue was originally created as [ARROW-17029](https://issues.apache.org/jira/browse/ARROW-17029). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the proposed rle_util.h utilities on the reporter's filter branch, then trace how physical and logical indices interact with Arrow offsets. The guide should cover converting scalar and vector kernels to RLE, including when to use internal data arrays or VisitMergedRuns, with those topics clearly explained for readers.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.