microsoft / microsoft/markitdown
Question: would a plugin-first offline OCR backend for scanned PDFs fit the current MarkItDown direction?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Hi maintainers, I have been exploring a small plugin-first OCR extension for MarkItDown and wanted to check whether this direction fits the project's intended boundaries before proposing any upstream changes.
Problem
MarkItDown already has:
- strong built-in support for machine-readable PDFs
- a plugin system for third-party converters
- an existing OCR plugin path based on LLM vision models
However, there still seems to be a practical gap for scanned PDFs in environments where users want:
- a local / offline OCR backend
- CPU-friendly deployment
- no dependency on an LLM API
- better coverage for Chinese report-style pages
This came up while testing Chinese report pages that mix:
- paragraphs
- chart labels and numbers
- image captions
- page footers / page numbers
One representative sample page includes both Chinese narrative text and embedded chart content in the same report page. I can provide redacted before/after output examples when sharing the plugin prototype.
Proposed direction
I am prototyping this as a separate plugin, not a core change:
- PDF-only in the first iteration
- opt-in only through
enable_plugins=True - keep built-in PDF behavior unchanged
- only trigger OCR fallback when the built-in PDF conversion returns empty content
This keeps the scope intentionally narrow and avoids changing default behavior.
Why plugin-first
The current plugin architecture already looks like the right extension point for OCR backends that are:
- optional
- dependency-heavy
- backend-specific
So my current plan is to keep this as a third-party plugin rather than proposing a large core integration.
Non-goals
- no changes to the default built-in PDF converter behavior
- no vendor-specific configuration added to
MarkItDown(...) - no attempt to replace the existing LLM-based OCR plugin
- no multi-format support in the first iteration
What I would like feedback on
- Does a plugin-first offline OCR backend for scanned PDFs fit the current project direction?
- If yes, would a small follow-up PR to improve docs around plugin-first OCR extensions be welcome?
- If there is any missing generic hook that would make third-party OCR plugins easier to implement, would maintainers prefer that as a separate very small PR?
If this direction looks reasonable, I can keep the OCR backend in a separate plugin package and only propose minimal upstream changes, if any, after the plugin is working.
Thanks.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing plugin architecture, built-in PDF conversion behavior, and the current LLM-based OCR plugin path. Clarify with maintainers whether an offline, PDF-only OCR plugin fits the project direction and whether documentation or a generic hook is wanted. Done means an agreed scope or follow-up issue, not an implementation started from this discussion.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100