zipfile.ZipFile fails to read zstd-compressed file consisting of multiple frames
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
I have a ZIP file archive.zip that contains a file data.npy. This file is compressed with zstd, consisting of two zstd frames. When I try to read this in with zipfile, I get a BadZipFile: Bad CRC-32 for file 'data.npy' error.
import zipfile
with zipfile.ZipFile("archive.zip", "r") as z:
with z.open("data.npy", "r") as f:
data = f.read() # -> BadZipFile: Bad CRC-32 ...
The error is a red herring, though: When I disable CRC checking, there is no exception thrown, but the data read is in fact only the data from the first zstd frame. The second frame gets ignored completely (and thus the checksum is wrong).
import zipfile
with zipfile.ZipFile("archive.zip", "r") as z:
with z.open("data.npy", "r") as f:
f._update_crc = lambda newdata: None
data = f.read() # -> result only contains data from first frame
As independent cross-check, I tested decompressing the file with both 7z and WinRAR without issues. So I don't think the file is somehow invalid. This is the file: archive.zip
CPython versions tested on:
3.14
Operating systems tested on:
Linux
Linked PRs
- gh-154537
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The reproducer uses zipfile.ZipFile.open and z.open on the attached archive; start by reproducing the two-frame zstd case in CPython's zipfile implementation. Done means reading data.npy returns both frames and the normal CRC check succeeds, with a regression test covering the behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100