[FEA] Deletion vector parquet reader should support return _deleted column.
Open
? - Needs Triage
feature request
- Dominant language
- C++
- Stars
- 9.8k
- Forks
- 1.1k
- Avg merge
- 3d 6m
- Merged PRs (30d)
- 278
Description
When reading from table format, sometimes we need to return _deleted metadata column to mask if a row has been deleted by deletion vector rather than filtering directly. For example, let say a parquet file has 5 rows, and the deletion vector deleted rows 1, 3, then when read from parquet, we should still return 5 rows, but with a bool column (F, T, F, T, F)
Contributor guide
Research direction
Start at the parquet reader path used for table-format reads and trace where deletion vectors currently filter rows. Check existing reader tests for metadata-column behavior; done means a five-row input with rows 1 and 3 deleted returns five rows plus _deleted values (F, T, F, T, F).
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100