microsoft / microsoft/markitdown

Feature Request: Add batch processing capability for directory conversion

Open
#1,371 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
186k
Forks
13.7k
Avg merge
1d 4h
Merged PRs (30d)
49

Description

User Story

As a user, I want to convert multiple files in a directory to Markdown format in one operation, so that I can efficiently process large collections of documents without having to run the command for each file individually.

Current Behavior

Currently, MarkItDown only supports converting single files:

markitdown path-to-file.pdf > document.md
Desired Behavior

I would like to be able to specify a directory and have MarkItDown process all supported files within that directory (and subdirectories):

# Process all files in a directory
markitdown --batch ./documents --output ./converted

# Process with specific file types
markitdown --batch ./documents --output ./converted --types pdf,docx,pptx

# Process with recursive subdirectory support
markitdown --batch ./documents --output ./converted --recursive
Use Cases
  1. Document Collections: Converting entire folders of PDFs, Word documents, and PowerPoint presentations
  2. Data Processing Pipelines: Batch processing for LLM training data preparation
  3. Documentation Migration: Converting legacy document collections to Markdown format
  4. Research Projects: Processing large datasets of various document types
Proposed Implementation
  1. Add --batch or -b flag to specify directory processing mode
  2. Add --output or -o flag to specify output directory
  3. Add --recursive or -r flag for subdirectory processing
  4. Add --types flag to filter by specific file extensions
  5. Maintain directory structure in output
  6. Provide progress reporting and error handling
Benefits
  • Efficiency: Reduce manual work for large document collections
  • Consistency: Ensure all files are processed with the same settings
  • Scalability: Handle large document repositories
  • Integration: Better integration with automated workflows
Alternative Solutions

Currently, users need to create custom scripts or use shell commands like:

find ./documents -name "*.pdf" -exec markitdown {} -o {}.md \;

This works but lacks the native integration and error handling that a built-in feature would provide.

Priority

Medium - This would significantly improve the user experience for users working with document collections.

Labels
  • enhancement
  • feature-request
  • cli
  • batch-processing

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by tracing the existing markitdown CLI command that converts a single file. Done means directory input supports the requested filtering, recursion, output-directory handling, structure preservation, progress reporting, and error handling described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
cli
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.