Nullus157 / Nullus157/async-compression
Expose `set_pledged_src_size` as a public API for `zstd` compression
- Dominant language
- Rust
- Stars
- 666
- Forks
- 118
- Avg merge
- 17h 18m
- Merged PRs (30d)
- 9
Description
### Issue
`async-compression` offers no way to call [`set_pledged_src_size()`](https://github.com/gyscos/zstd-rs/blob/739b040997dca5899cafef3b3be73cd5adf4a651/src/stream/raw.rs#L355) on the `zstd` encoder. Could we provide a public API in `async-compression` to call this?
### Context
`zstd` encoding can include an optional [`Frame_Content_Size` value](https://github.com/facebook/zstd/blob/dev/doc/zstd_compression_format.md#frame_content_size). Decompression libraries such as Python's [`zstandard`](https://python-zstandard.readthedocs.io/en/latest/) provide APIs (e.g. [`decompress`](https://python-zstandard.readthedocs.io/en/latest/decompressor.html#zstandard.ZstdDecompressor.decompress)) that use this value to decompress the data in a single operation. Without it, these libraries must treat the encoded bytes as a stream.
Writing `Frame_Content_Size` requires two steps (from the [`zstd` manual](https://facebook.github.io/zstd/zstd_manual.html)):
1. Enable `ZSTD_c_contentSizeFlag`. `async-compression` already offers [a `CParameter` method](https://docs.rs/async-compression/latest/async_compression/zstd/struct.CParameter.html#method.content_size_flag) to do this.
2. Call `ZSTD_CCtx_setPledgedSrcSize()` to set the decompressed data size. The `zstd` crate already exposes the public method [`set_pledged_src_size()`](https://github.com/gyscos/zstd-rs/blob/739b040997dca5899cafef3b3be73cd5adf4a651/src/stream/raw.rs#L355) for this, but it's not exposed by the `async-compression` crate.
When `set_pledged_src_size` is not called, the encoded bytes won't have a `Frame_Content_Size` value.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.