Allow for custom compression codecs
- Lingua principale
- Java
- Stelle
- 3.1k
- Fork
- 1.6k
- Merge medio
- 3g 12h
- PR unite (30g)
- 33
Descrizione
I understand that the list of accepted compression codecs is explicity limited to uncompressed, snappy, gzip, and lzo. (See parquet.hadoop.metadata.CompressionCodecName.java) Is there a reason for this? Or is there an easy workaround? On the surface it seems like an unnecessary restriction.
I ask because I have written a custom codec to implement encryption and I'm unable to use it with Parquet, which is a real shame because it is the main storage format I was hoping to use.
Other thoughts on how to implement encryption in Parquet with this limitation?
**Reporter**: [Steven Anton](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=santon)
**Note**: *This issue was originally created as [PARQUET-678](https://issues.apache.org/jira/browse/PARQUET-678). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Direzione di ricerca
Start with parquet.hadoop.metadata.CompressionCodecName.java, the file named in the issue, and review the migration documentation linked in the issue note. Clarify whether the goal is custom codec support, an encryption workaround, or both; done should be a decided and documented approach that permits the requested use case.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- java
- Ambito
- data-engineering
- Tipo di issue
- Funzionalità
- Difficoltà
- 5/5
- Tempo stimato
- Più di una settimana
- Stato di attività
- Ferma
- Chiarezza
- Da chiarire
- Idoneità per principianti
- 25/100