indygreg / indygreg/python-zstandard

Can't decompress data which is compressed by rust code

Open
#235 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C
Stars
642
Forks
116
Avg merge
1d 14h
Merged PRs (30d)
5

Description

Here is simplest reproducing example:

import zstandard
zstandard.decompress(b'\x28\xb5\x2f\xfd\x00\x58\x11\x00\x00\x7b\x7d')

It raises an error: ZstdError: could not determine content size in frame header

More context

I'm trying to rewrite a client application in rust, it sends compressed data to server, then server decompresses it. Unfortunally the server failed to decompress data.

Here is how I do it in client side:

use std::io::Cursor;
use zstd;

fn main() {
    let body = zstd::encode_all(Cursor::new("{}".as_bytes()), 3).unwrap();
    for x in body.iter() {
        print!("\\x{x:x?}");
    }
}

And I copied the body and decompressed it in python, and it failed.


If I tried to compress data({} in my example) in python, and decompressed in rust, it successes. So I think it's the issue in python side.

In rust, I'm using zstd-rs for compressing/decompressing

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure through zstandard.decompress with the supplied frame, then compare it with Rust zstd::encode_all and the reverse Python/Rust path. Done means determining the expected handling of frames without a content size and adding a regression test or documenting the required decompression input.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.