microsoft / microsoft/markitdown
Feature Request: Add batch processing capability for directory conversion
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
User Story
As a user, I want to convert multiple files in a directory to Markdown format in one operation, so that I can efficiently process large collections of documents without having to run the command for each file individually.
Current Behavior
Currently, MarkItDown only supports converting single files:
markitdown path-to-file.pdf > document.md
Desired Behavior
I would like to be able to specify a directory and have MarkItDown process all supported files within that directory (and subdirectories):
# Process all files in a directory
markitdown --batch ./documents --output ./converted
# Process with specific file types
markitdown --batch ./documents --output ./converted --types pdf,docx,pptx
# Process with recursive subdirectory support
markitdown --batch ./documents --output ./converted --recursive
Use Cases
- Document Collections: Converting entire folders of PDFs, Word documents, and PowerPoint presentations
- Data Processing Pipelines: Batch processing for LLM training data preparation
- Documentation Migration: Converting legacy document collections to Markdown format
- Research Projects: Processing large datasets of various document types
Proposed Implementation
- Add
--batchor-bflag to specify directory processing mode - Add
--outputor-oflag to specify output directory - Add
--recursiveor-rflag for subdirectory processing - Add
--typesflag to filter by specific file extensions - Maintain directory structure in output
- Provide progress reporting and error handling
Benefits
- Efficiency: Reduce manual work for large document collections
- Consistency: Ensure all files are processed with the same settings
- Scalability: Handle large document repositories
- Integration: Better integration with automated workflows
Alternative Solutions
Currently, users need to create custom scripts or use shell commands like:
find ./documents -name "*.pdf" -exec markitdown {} -o {}.md \;
This works but lacks the native integration and error handling that a built-in feature would provide.
Priority
Medium - This would significantly improve the user experience for users working with document collections.
Labels
enhancementfeature-requestclibatch-processing
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the existing markitdown CLI command that converts a single file. Done means directory input supports the requested filtering, recursion, output-directory handling, structure preservation, progress reporting, and error handling described in the issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100