NVIDIA / NVIDIA/cudf

[FEA] Deletion vector parquet reader should support return _deleted column.

Open
#23,774 3 comments 0 reactions 0 assignees View on GitHub
? - Needs Triage feature request
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

When reading from table format, sometimes we need to return _deleted metadata column to mask if a row has been deleted by deletion vector rather than filtering directly. For example, let say a parquet file has 5 rows, and the deletion vector deleted rows 1, 3, then when read from parquet, we should still return 5 rows, but with a bool column (F, T, F, T, F)

Contributor guide

Open the contributing guide

Research direction

Start at the parquet reader path used for table-format reads and trace where deletion vectors currently filter rows. Check existing reader tests for metadata-column behavior; done means a five-row input with rows 1 and 3 deleted returns five rows plus _deleted values (F, T, F, T, F).

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.