Nullus157 / Nullus157/async-compression

GzipDecoder read_lines terminated early, but fixes on deleting an empty line?!

Open
#153 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
666
Forks
118
Avg merge
17h 18m
Merged PRs (30d)
9

Description

I am having a weird interaction with async_compression::tokio::bufread::GzipDecoder and tokio::io::AsyncBufReadExt. This [example](https://gist.github.com/be22f6143481e12332f0c432f3fb4c01) shows that when I use AsyncBufReadExt to read the number of lines in an unzipped file I get 413484 lines, but if I use a Bufreader wrapped around a GzipDecoder on a gzipped version of the same file I only get 65654 lines. I can fix this error by removing an empty line somewhere before the divergence point, at which point both files will report 413483 lines. This makes me think there is some edge-case with the various buffers that cause the GzipDecoder read_lines to terminate early, and any small change (removing that one empty line) manages to get things working again. I can't share the files but would be happy to diagnose further if anyone has suggestions.

EDIT: This error does not occur if I use the synchronous flate2 decompression by the way, so it is something specific to the tokio/async_compression interactions.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.