Nullus157 / Nullus157/async-compression

Gzip file created with async compression not decodable

Open
#135 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
666
Forks
118
Avg merge
17h 18m
Merged PRs (30d)
9

Description

Hello,

I try to create a stream compressed gzip file on the fly while receiving chunks of data.
My issue is that the file cannot be decoded from gzip/gunzip :-(

I stripped it down to a test encoding that just encodes "test" to see what the difference is compared to a Node.js based test encoding that works. But I do not get why there is a difference, and if this is really a bug inside the library, or an issue of how i use the async file io.

Here is my sample code:

```
use tokio::fs::File;
use async_compression::tokio_02::write::{ GzipEncoder };

let mut file = File::create(„test.txt.gz“).await?;
let writer = GzipEncoder::new(file);
writer.write("test".as_bytes()).await?;
```
Result:
```
xxd test.txt.gz
00000000: 1f8b 0800 0000 0000 00ff 2a49 2d2e 0100 ..........*I-...
00000010: 0000 ffff ....
```

This file is not decodable! Gunzip says it is corrupt!

When I create the file via this small node.js script
```
var gz = zlib.createGzip(); // createGzip
gz.pipe(fs.createWriteStream(„test_node.txt.gz“));
gz.write(„test“);
gz.end()
```
the resulting file has the following content:
```
00000000: 1f8b 0800 0000 0000 0013 2b49 2d2e 0100 ..........+I-...
00000010: 0c7e 7fd8 0400 0000 .~......
```

The issue is not the missing checksum and file length:
I added checksum and length via gzip-header create. The file is still not decodable via gunzip

The interesting bit seems to be the encoded stream, they differ between async-compression and the working Node.js:
RUST: 2a49 2d2e 0100 0000 ffff
Node: 2b49 2d2e 0100

Why are there 4 tailing bytes 0000 ffff ?
And why is the first byte different?

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.