facebook / facebook/zstd

zstdcat should return all decompressable data even if frame is incomplete

Open
#3,602 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
C
Stars
27.9k
Forks
2.6k
Avg merge
1d 3h
Merged PRs (30d)
8

Description

This issue manifests as truncated data when using zstdcat on some files that have an unterminated frame.

When the frame is unterminated, `ZSTD_decompressStream` does not keep [hostage input bytes](https://github.com/facebook/zstd/blob/838f96a9559d431d8d57d5883d7b67fd8df25ada/lib/decompress/zstd_decompress.c#L2324). In circumstances where the input buffer has been cleared, but [not all data have been written to the output buffer](https://github.com/facebook/zstd/blob/838f96a9559d431d8d57d5883d7b67fd8df25ada/lib/zstd.h#L885), zstdcat should continue to call `ZSTD_decompressStream` to write out all data that's been decompressed before erroring out.
https://github.com/facebook/zstd/blob/c28031df8f1809621407b5bc9c4b3e052872409f/programs/fileio.c#L2391

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.