apache / apache/parquet-java

Data obfuscation layer for encryption

Open
#2,211 0 comments 0 reactions 0 assignees View on GitHub
Component: Parquet Priority: Major Type: enhancement
Dominant language
Java
Stars
3.1k
Forks
1.6k
Avg merge
3d 12h
Merged PRs (30d)
33

Description

Data obfuscation in sensitive columns - for users without access to column encryption keys.
1. Implement on top of [basic Parquet encryption](https://github.com/apache/parquet-format/blob/encryption/Encryption.md) 
1. Built-in support for multiple masking mechanisms, with different trade-off between data utility, leakage, and size/throughput overhead
1. Provide interface for plug-in custom masking mechanism
1. Enable storing multiple masked versions of the same column in a file
1. Provide readers with explicit list of column’s masked versions in a file
1. Enable readers to select a masked version of a column
1. Stretch: Implement tools for analysis of file data privacy properties and information leakage
1. Stretch: Leverage privacy analysis tools for tuning file data anonymity
1. Optional: Support aggregated obfuscation

**Reporter**: [Gidon Gershinsky](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=gershinsky) / @ggershinsky
**Assignee**: [Gidon Gershinsky](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=gershinsky) / @ggershinsky
#### Related issues:
- [Parquet modular encryption](https://github.com/apache/parquet-java/issues/2110) (depends upon)
#### PRs and other links:
- [discussion googledoc](https://docs.google.com/document/d/1LMs74uhqvMNJacBySPnWq6tM8qIpgcIZz444c7vfibM/edit?usp=sharing)

**Note**: *This issue was originally created as [PARQUET-1376](https://issues.apache.org/jira/browse/PARQUET-1376). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reading the linked Parquet basic encryption specification and the related modular encryption issue, then review the linked discussion document. The issue describes masking mechanisms, plug-in support, multiple masked column versions, reader selection, and optional analysis tools, but names no Java files or tests; done would require agreeing on and implementing that broad design.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
cryptography, data-engineering, security
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.