indygreg / indygreg/python-zstandard
No error thrown when there is garbage data past end-of-frame
Nobody has claimed this yet.
- Dominant language
- C
- Stars
- 642
- Forks
- 116
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 5
Description
On 0.17.0, the following example doesn't throw an error:
>>> zstandard.decompress(zstandard.compress(b"foo") + b"garbage")
b'foo'
Perhaps debatable, but I think an error should always be thrown if there is data past the last valid frame in a stream. Especially after fixing https://github.com/indygreg/python-zstandard/issues/59, I don't think it makes sense to silently ignore the garbage past end-of-frame. Users should be notified if they are constructing invalid streams.
Thanks @thatch for identifying this issue.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the example using zstandard.decompress and zstandard.compress with trailing garbage, then trace the decompression entry point to understand how bytes after the valid frame are handled. Done means trailing data raises an error while the valid compressed frame still decompresses successfully; add a regression test covering this behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100