gzip compression not backwards compatible with gzip 1.3.12
Nobody has claimed this yet.
- Dominant language
- Scala
- Stars
- 2.5k
- Forks
- 636
- Avg merge
- 2d 4h
- Merged PRs (30d)
- 7
Description
Gzip compressed files produced with fs2.compression.gzip are not recognized by gzip 1.3.12. This is primarily an issue for users on CentOS 6 (CentOS 7 ships a newer version that handles this).
Here's how to reproduce:
Produce a compressed file:
import java.nio.file.Paths
import cats.effect.{Blocker, ExitCode, IO, IOApp}
import cats.syntax.all._
object WriteCompressedFile extends IOApp {
override def run(args: List[String]): IO[ExitCode] = Blocker[IO].use { b =>
fs2.Stream.emit("lorem ipsum")
.covary[IO]
.through(fs2.text.utf8Encode)
.through(fs2.compression.gzip())
.through(fs2.io.file.writeAll(Paths.get("/tmp/bar/foo.gz"), b)).compile.drain.as(ExitCode.Success)
}
}
Uncompress on CentOS 6
├─ /tmp/bar
╰ file foo.gz
foo.gz: gzip compressed data, has CRC, from MacOS, original size modulo 2^32 37
├─ /tmp/bar
╰ docker run --rm -it -v $(pwd):/foo centos:centos6 bash
[root@52c59137bb4b /]# cd foo
[root@52c59137bb4b foo]# ls -l
total 4
-rw-r--r-- 1 root root 57 Jun 5 08:21 foo.gz
[root@52c59137bb4b foo]# file foo.gz
foo.gz: gzip compressed data, has CRC, from MacOS
[root@52c59137bb4b foo]# gunzip foo.gz
gzip: foo.gz is a a multi-part gzip file -- not supported
[root@52c59137bb4b foo]# gzip -d foo.gz
gzip: foo.gz is a a multi-part gzip file -- not supported
[root@52c59137bb4b foo]#
Related issue on CentOS:
- https://serverfault.com/questions/727752/how-to-upgrade-gzip-from-1-3-12-to-1-6-on-centos-6
- https://forums.centos.org/viewtopic.php?p=260691#p260691
Doesn't seem likely to change there (snipped from last link):
What we ship is 1.3.12 and will be for the lifespan of the distro. CentOS 7 uses 1.5.8 and will do for its life too.
Not sure if it's feasible to be backwards compatible with gzip à 2007, but it's good to document incompatibilities anyway.
Feel free to close if this is out of scope
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the fs2.compression.gzip entry point and reproduce the generated file with the Scala example, then test it with gzip 1.3.12 in the CentOS 6 environment described. Determine whether compatibility can be supported; if not, document the incompatibility and its affected versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- scala
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100