zipfiles.ZipFile's internal ZipInfo entry cache has no internal bounds
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
Bug report
Bug description:
ZIP files with substantially many entries will exhaust memory when initialized using the zipfile module. As far as I know, there is no way to prevent this using the zipfile.ZipFile API.
The following code produces a ZIP file of 427 MB:
import zipfile
import tracemalloc
N = 5_000_000
tracemalloc.start()
try:
with zipfile.ZipFile("test.zip", "w") as zf:
for i in range(N):
zf.writestr(str(i), "")
finally:
m, _ = tracemalloc.get_traced_memory()
print(f"{m >> 20} MB")
The following code on the produced ZIP file outputs 2408 MB on my machine:
import zipfile
import tracemalloc
tracemalloc.start()
try:
with zipfile.ZipFile("test.zip", "r") as zf:
pass
finally:
m, _ = tracemalloc.get_traced_memory()
print(f"{m >> 20} MB")
CPython versions tested on:
CPython main branch, 3.14
Operating systems tested on:
Linux
Linked PRs
- gh-153465
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the zipfile.ZipFile entry point and run the reported tracemalloc reproducer against a ZIP containing many entries. Review the linked PR gh-153465 before starting, since the work may already be underway. Done means initialization no longer allows the internal ZipInfo entry cache to exhaust memory, with coverage for the reported large-entry case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 20/100