microsoft / microsoft/markitdown

Make ZipConverter safety limits configurable via constructor parameters

Open Beginner friendly
#1,661 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
186k
Forks
13.7k
Avg merge
1d 4h
Merged PRs (30d)
49

Description

Problem

PR #1628 added zip bomb protection to ZipConverter with three hardcoded module-level constants:

  • MAX_DECOMPRESSED_FILE_SIZE = 100 MB per file
  • MAX_DECOMPRESSION_RATIO = 100:1
  • MAX_TOTAL_DECOMPRESSED_SIZE = 500 MB total

These defaults are reasonable for general use, but they are not configurable. This creates a real problem for legitimate use cases:

  • Scientific datasets: ZIP archives commonly contain files well over 100 MB (genomics data, satellite imagery, simulation outputs).
  • Legal and financial document archives: SEC EDGAR bulk data packages and court document bundles regularly exceed 500 MB total.
  • Internal tooling: An organization running markitdown on known-safe internal archives has no way to raise the limits without monkey-patching the module.

Users who hit these limits get silent skipping with only a logger warning, and no way to know programmatically that their content was truncated.

Proposed solution

Move the limits to constructor parameters on ZipConverter with the current values as defaults:

class ZipConverter(DocumentConverter):
    def __init__(
        self,
        max_file_size: int = 100 * 1024 * 1024,
        max_ratio: int = 100,
        max_total_size: int = 500 * 1024 * 1024,
    ):
        self.max_file_size = max_file_size
        self.max_ratio = max_ratio
        self.max_total_size = max_total_size

This is a non-breaking change: the defaults stay the same, and users who need larger limits can pass them explicitly. Users who want to disable the limits entirely can pass float("inf").

Additional context

This was flagged during review of PR #1628 as a blocker before merge. Opening as a tracked issue so it does not get lost if the PR merges first.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the ZipConverter entry point and inspect where the three module-level safety limits are defined and used. Make the existing values available as constructor defaults and ensure explicit values control the limits without changing default behavior; verify with the repository’s existing ZIP-conversion tests if available.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
tooling
Issue type
Feature
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.