Allow for custom compression codecs
- Vorherrschende Sprache
- Java
- Sterne
- 3.1k
- Forks
- 1.6k
- Ø Merge
- 3 T. 12 Std.
- Gemergte PRs (30 T.)
- 33
Beschreibung
I understand that the list of accepted compression codecs is explicity limited to uncompressed, snappy, gzip, and lzo. (See parquet.hadoop.metadata.CompressionCodecName.java) Is there a reason for this? Or is there an easy workaround? On the surface it seems like an unnecessary restriction.
I ask because I have written a custom codec to implement encryption and I'm unable to use it with Parquet, which is a real shame because it is the main storage format I was hoping to use.
Other thoughts on how to implement encryption in Parquet with this limitation?
**Reporter**: [Steven Anton](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=santon)
**Note**: *This issue was originally created as [PARQUET-678](https://issues.apache.org/jira/browse/PARQUET-678). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Rechercherichtung
Start with parquet.hadoop.metadata.CompressionCodecName.java, the file named in the issue, and review the migration documentation linked in the issue note. Clarify whether the goal is custom codec support, an encryption workaround, or both; done should be a decided and documented approach that permits the requested use case.
Vom Indexierungsmodell aus dem Issue-Text verfasst.
Bewertung
- Tech-Stack
- java
- Bereich
- data-engineering
- Issue-Typ
- Feature
- Schwierigkeit
- 5/5
- Geschätzter Aufwand
- Über eine Woche
- Aktivitätsstatus
- Veraltet
- Klarheit
- Muss geklärt werden
- Anfängerfreundlichkeit
- 25/100