NVIDIA / NVIDIA/cudf

[FEA] Parquet Modular Encryption support

Open
#23,619 2 comments 0 reactions 0 assignees View on GitHub
feature request
Dominant language
C++
Stars
9.8k
Forks
1.1k
Avg merge
3d 6m
Merged PRs (30d)
278

Description

**Is your feature request related to a problem? Please describe.**

Apache Iceberg V3 supports encryption of table data, and it utilizes parquet's modular encryption to encrypt data files under the hood.

**Describe the solution you'd like**

To support read and write encrypted iceberg tables, we need to support read/write encrypted parquet files. The scope of this issue is not full feature support of PME, since iceberg only utilized full file encryption. To put it in simple, iceberg will manage the encryption and aad for reading/writing a parquet file.

Contributor guide

Open the contributing guide

Research direction

The issue names no files, tests, or entry points. Start by locating cuDF's Parquet read and write paths and determine how full-file modular encryption, encryption, and AAD would integrate; done means encrypted Parquet files can be read and written for Iceberg's stated scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
data-engineering
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.