indygreg / indygreg/python-zstandard
Can't decompress data which is compressed by rust code
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 642
- Forks
- 116
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 5
Description
Here is simplest reproducing example:
import zstandard
zstandard.decompress(b'\x28\xb5\x2f\xfd\x00\x58\x11\x00\x00\x7b\x7d')
It raises an error: ZstdError: could not determine content size in frame header
More context
I'm trying to rewrite a client application in rust, it sends compressed data to server, then server decompresses it. Unfortunally the server failed to decompress data.
Here is how I do it in client side:
use std::io::Cursor;
use zstd;
fn main() {
let body = zstd::encode_all(Cursor::new("{}".as_bytes()), 3).unwrap();
for x in body.iter() {
print!("\\x{x:x?}");
}
}
And I copied the body and decompressed it in python, and it failed.
If I tried to compress data({} in my example) in python, and decompressed in rust, it successes. So I think it's the issue in python side.
In rust, I'm using zstd-rs for compressing/decompressing
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the failure through zstandard.decompress with the supplied frame, then compare it with Rust zstd::encode_all and the reverse Python/Rust path. Done means determining the expected handling of frames without a content size and adding a regression test or documenting the required decompression input.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100