microsoft / microsoft/markitdown
Make ZipConverter safety limits configurable via constructor parameters
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Problem
PR #1628 added zip bomb protection to ZipConverter with three hardcoded module-level constants:
MAX_DECOMPRESSED_FILE_SIZE = 100 MBper fileMAX_DECOMPRESSION_RATIO = 100:1MAX_TOTAL_DECOMPRESSED_SIZE = 500 MBtotal
These defaults are reasonable for general use, but they are not configurable. This creates a real problem for legitimate use cases:
- Scientific datasets: ZIP archives commonly contain files well over 100 MB (genomics data, satellite imagery, simulation outputs).
- Legal and financial document archives: SEC EDGAR bulk data packages and court document bundles regularly exceed 500 MB total.
- Internal tooling: An organization running markitdown on known-safe internal archives has no way to raise the limits without monkey-patching the module.
Users who hit these limits get silent skipping with only a logger warning, and no way to know programmatically that their content was truncated.
Proposed solution
Move the limits to constructor parameters on ZipConverter with the current values as defaults:
class ZipConverter(DocumentConverter):
def __init__(
self,
max_file_size: int = 100 * 1024 * 1024,
max_ratio: int = 100,
max_total_size: int = 500 * 1024 * 1024,
):
self.max_file_size = max_file_size
self.max_ratio = max_ratio
self.max_total_size = max_total_size
This is a non-breaking change: the defaults stay the same, and users who need larger limits can pass them explicitly. Users who want to disable the limits entirely can pass float("inf").
Additional context
This was flagged during review of PR #1628 as a blocker before merge. Opening as a tracked issue so it does not get lost if the PR merges first.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the ZipConverter entry point and inspect where the three module-level safety limits are defined and used. Make the existing values available as constructor defaults and ensure explicit values control the limits without changing default behavior; verify with the repository’s existing ZIP-conversion tests if available.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100