munzip fails on gzipped files
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 84
- Forks
- 12
- PR merge metrics
- No merged PRs in 30d
Description
Hi guys,
Thanks for the great library! It's made some of my Java programs that were I/O bound much faster :)
However, it's currently hard to use MiGz unless you can guarantee that it will never try to unzip a file zipped by vanilla `gzip`. If you do, it crashes:
```
% gzip junk.csv
% munzip < junk.csv.gz > junk.csv
Decompressing stdin using 40 threads
Exception in thread "main" java.lang.RuntimeException: java.util.zip.DataFormatException: invalid code lengths set
at com.linkedin.migz.MiGzInputStream.decompressorThreadWithInflater(MiGzInputStream.java:208)
at com.linkedin.migz.MiGzInputStream.decompressorThread(MiGzInputStream.java:114)
at com.concurrentli.Interrupted.lambda$ignored$1(Interrupted.java:48)
at java.base/java.lang.Thread.run(Thread.java:830)
Caused by: java.util.zip.DataFormatException: invalid code lengths set
at java.base/java.util.zip.Inflater.inflateBytesBytes(Native Method)
at java.base/java.util.zip.Inflater.inflate(Inflater.java:378)
at java.base/java.util.zip.Inflater.inflate(Inflater.java:464)
at com.linkedin.migz.MiGzInputStream.decompressorThreadWithInflater(MiGzInputStream.java:192)
... 3 more
```
Would it be possible for MiGz to just decompress serially in this case? The opposite seems to work fine; I can decompress MiGz files with gunzip with no problem.
This is MiGz 1.0.1 running under Oracle OpenJDK 13.0.2 on RedHat 7.4.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure with the reported gzip and munzip stdin/stdout commands, then inspect MiGzInputStream.java, especially decompressorThread and decompressorThreadWithInflater. The change is complete when vanilla gzip input no longer throws DataFormatException and is decompressed serially as requested.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, cli
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 45/100