microsoft / microsoft/markitdown

Question: would a plugin-first offline OCR backend for scanned PDFs fit the current MarkItDown direction?

Open
#1,650 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
186k
Forks
13.7k
Avg merge
1d 4h
Merged PRs (30d)
49

Description

Hi maintainers, I have been exploring a small plugin-first OCR extension for MarkItDown and wanted to check whether this direction fits the project's intended boundaries before proposing any upstream changes.

Problem

MarkItDown already has:

  • strong built-in support for machine-readable PDFs
  • a plugin system for third-party converters
  • an existing OCR plugin path based on LLM vision models

However, there still seems to be a practical gap for scanned PDFs in environments where users want:

  • a local / offline OCR backend
  • CPU-friendly deployment
  • no dependency on an LLM API
  • better coverage for Chinese report-style pages

This came up while testing Chinese report pages that mix:

  • paragraphs
  • chart labels and numbers
  • image captions
  • page footers / page numbers

One representative sample page includes both Chinese narrative text and embedded chart content in the same report page. I can provide redacted before/after output examples when sharing the plugin prototype.

Proposed direction

I am prototyping this as a separate plugin, not a core change:

  • PDF-only in the first iteration
  • opt-in only through enable_plugins=True
  • keep built-in PDF behavior unchanged
  • only trigger OCR fallback when the built-in PDF conversion returns empty content

This keeps the scope intentionally narrow and avoids changing default behavior.

Why plugin-first

The current plugin architecture already looks like the right extension point for OCR backends that are:

  • optional
  • dependency-heavy
  • backend-specific

So my current plan is to keep this as a third-party plugin rather than proposing a large core integration.

Non-goals

  • no changes to the default built-in PDF converter behavior
  • no vendor-specific configuration added to MarkItDown(...)
  • no attempt to replace the existing LLM-based OCR plugin
  • no multi-format support in the first iteration

What I would like feedback on

  1. Does a plugin-first offline OCR backend for scanned PDFs fit the current project direction?
  2. If yes, would a small follow-up PR to improve docs around plugin-first OCR extensions be welcome?
  3. If there is any missing generic hook that would make third-party OCR plugins easier to implement, would maintainers prefer that as a separate very small PR?

If this direction looks reasonable, I can keep the OCR backend in a separate plugin package and only propose minimal upstream changes, if any, after the plugin is working.

Thanks.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the existing plugin architecture, built-in PDF conversion behavior, and the current LLM-based OCR plugin path. Clarify with maintainers whether an offline, PDF-only OCR plugin fits the project direction and whether documentation or a generic hook is wanted. Done means an agreed scope or follow-up issue, not an implementation started from this discussion.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend, tooling
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.