apache / apache/arrow

[C++][Parquet] Support limit RowGroup size by bytes.

Open
#50,704 2 comments 0 reactions 0 assignees View on GitHub
Component: C++ Type: enhancement
Dominant language
C++
Stars
17.1k
Forks
4.3k
Avg merge
3d 13h
Merged PRs (30d)
88

Description

### Describe the enhancement requested

Could we support limit RowGroup size by bytes (before compression) in addition to RowGroup length (records count) to provide more precisely writing control.

### Component(s)

C++

Contributor guide

Open the contributing guide

Research direction

Begin in the C++ Parquet row-group writing path and compare the existing record-count limit. Identify the public control and relevant writer tests, then verify that a byte-based limit applies to uncompressed row-group data while existing record-count behavior remains intact.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.