apache / apache/arrow

[C++][Parquet] Generalize existing null bitmap generation

Open
#25,853 1 comment 0 reactions 0 assignees View on GitHub
Component: C++ Type: task
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

Right now null bitmap generation assumes only list nesting.  Generalize and refactor exisitn code without changing existing functionality to accept additional parameters to support arrow nested types:

 

1.  Repeated ancestor def level

2.  Null slot usage (for fixed size lists)

 

 

**Reporter**: [Micah Kornfield](https://issues.apache.org/jira/browse/ARROW-9810) / @emkornfield
#### PRs and other links:
- [GitHub Pull Request #8156](https://github.com/apache/arrow/pull/8156)

**Note**: *This issue was originally created as [ARROW-9810](https://issues.apache.org/jira/browse/ARROW-9810). Please see the [migration documentation](https://github.com/apache/arrow/issues/14542) for further details.*

Contributor guide

Open the contributing guide

Research direction

Review the existing null bitmap generation for list nesting and compare the requested behavior with pull request #8156. The work is done when repeated ancestor definition levels and null slot usage for fixed-size lists are supported without changing existing functionality.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.