Sample of usage Parquet-1396 and Parquet-1178 for column level encryption with pluggable key access
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
This Jira provides a sample to use Parquet-1396 and Parquet-1178 column level encryption with pluggable key access. The Spark SQL application shows how to configure Parquet-1396 to encrypt and columns. The project CryptoMetadataRetriever shows how to implement the interface defined in Parquet-1396 as an example.
The project to be uploaded soon.
**Reporter**: [Xinli Shang](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=shangx@uber.com) / @shangxinli
#### Related issues:
- [EncryptionPropertiesFactory and DecryptionPropertiesFactory](https://github.com/apache/parquet-java/issues/2220) (Blocked)
- [Passing Field Metadata to Parquet](https://issues.apache.org/jira/browse/SPARK-25858) (Blocked)
- [Parquet modular encryption](https://github.com/apache/parquet-java/issues/2110) (depends upon)
#### Externally tracked issue: [https://github.com/apache/parquet-java/issues/2220](https://github.com/apache/parquet-java/issues/2220)
**Note**: *This issue was originally created as [PARQUET-1397](https://issues.apache.org/jira/browse/PARQUET-1397). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reading Parquet-1396 and Parquet-1178, then review the blocked related issues 2220 and SPARK-25858 and the dependent issue 2110. The issue says a Spark SQL application and a CryptoMetadataRetriever project should demonstrate encryption configuration and the interface implementation, but the project is not yet attached. Done means the sample project is uploaded and documents both usage paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, spark
- Domain
- data-engineering, security
- Issue type
- Documentation
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100