Make BZIP compressed files splittable
- Dominant language
- Java
- Stars
- 8.7k
- Forks
- 4.7k
- Avg merge
- 2d 2h
- Merged PRs (30d)
- 205
Description
Bzip2 is compressed as blocks, so it should be possible to do dynamic splitting. To do this: Seek to a location in the bzip, then keep seeking until you find the 6 byte block-start sequence 0x314159265359 (which is the 12 digit approximation of pi). You can use a bzip2 decompressor from that point onwards.
Imported from Jira [BEAM-683](https://issues.apache.org/jira/browse/BEAM-683). Original Jira may contain additional context.
Reported by: tim1357.
Contributor guide
Research direction
No file, test, or entry point is named; first locate Beam's BZIP input implementation and any existing split-related tests. Done means seeking from a split position finds the six-byte block-start sequence and decompression can continue from that block.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- data
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100