bzip2 compression
- Dominant language
- Java
- Stars
- 3.1k
- Forks
- 1.6k
- Avg merge
- 3d 12h
- Merged PRs (30d)
- 33
Description
Hi,
I have a requirement to implement Parquet with bzip2 compression because it's splitable. Right now, we can't provide bzip2 in PIG.
SET parquet.compression none/gzip/SNAPPY;
Is there any way to compress to bzip2 on top parquet ?
**Reporter**: [Rajasekhar Konda](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=rkonda)
**Note**: *This issue was originally created as [PARQUET-1011](https://issues.apache.org/jira/browse/PARQUET-1011). Please see the [migration documentation](https://issues.apache.org/jira/browse/PARQUET-2502) for further details.*
Contributor guide
No contributing guide indexed for this repository
Research direction
No files or tests are named. Start by tracing the existing parquet.compression setting and how PIG passes codec choices; then determine the integration needed for bzip2. Done means bzip2 is accepted alongside none, gzip, and SNAPPY and works for the requested PIG use case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data-engineering
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100