microsoft / microsoft/markitdown
markitdown-ocr: three PDF bugs — scanned pages never OCR'd, per-page PDF reopen, page-level scan misdetection
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Found while converting a real corpus of regulatory PDFs (66 files, mixed text/scanned) with markitdown-ocr 0.1.0. Three independent bugs in packages/markitdown-ocr/src/markitdown_ocr/_pdf_converter_with_ocr.py, all reproducible on current main.
Bug 1 (severe): scanned-PDF full-page OCR never triggers
The final fallback checks if ocr_service and (not markdown or not markdown.strip()), but the page loop appends \n## Page N\n\n markers for every page before content extraction. As a result markdown is never empty for any multi-page PDF, so _ocr_full_pages() never runs and scanned PDFs silently produce only page markers.
Fix: strip page markers before the emptiness check, e.g. content = re.sub(r'\n## Page \d+\n', '', markdown).strip().
Bug 2 (perf): _extract_page_images() reopens the whole PDF once per page
The page loop already holds an open pdfplumber.open(pdf_bytes) as pdf, but each page then calls _extract_page_images(pdf_bytes, page_num) which does pdfplumber.open(pdf_bytes) again — N full parses for an N-page document. On an 89-page / 15 MB standard this added ~30% wall time.
Fix: pass the already-open pdf object instead of pdf_bytes (one-line signature + call-site change). Measured: 1260s → 896s on that file.
Bug 3 (severe, data loss): page-level scan misdetection loses all body text
When a page has zero extractable text but contains small decorative images (logos/icons — common in web-page-print PDFs), the code takes the embedded-image path: those tiny images get OCR'd ("image contains no text") while the actual scanned page content is never rendered/OCR'd. The document-level fallback (Bug 1's check) doesn't fire because the page markers + junk OCR snippets make the output non-empty. In our sample this hit 3 of 7 scanned files (43%) with near-total content loss (e.g. 3 pages → 189 chars; after fix → 9,549 chars).
Fix: page-level check at the top of the loop — if (page.extract_text() or "").strip() is empty, render the full page at 300 DPI and OCR it (same code path as _ocr_full_pages), skipping the embedded-image path for that page.
Environment / validation
- markitdown 0.1.7 + markitdown-ocr 0.1.0, Python 3.13
- All three fixes applied locally and validated against the corpus (0 empty outputs, 0 failures after fixes)
- Happy to open a PR with the patches if helpful — they're small and localized to this file.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in packages/markitdown-ocr/src/markitdown_ocr/_pdf_converter_with_ocr.py and inspect the page loop, _extract_page_images(), and _ocr_full_pages(). Verify the three fixes against mixed text/scanned PDFs, including the 89-page performance case and pages with decorative images. Done means scanned pages are OCR'd, the PDF is not reopened per page, and the supplied corpus has no empty outputs or failures.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100