firecrawl / firecrawl/pdf-inspector
Dense invisible OCR text layers (Tr 3) are classified TextBased and skip the invisible-text fallback — "OCR is required" on OCRmyPDF output
- Dominant language
- Rust
- Stars
- 19.1k
- Forks
- 1.3k
- Avg merge
- 9h 21m
- Merged PRs (30d)
- 51
Description
## Summary
PDFs whose only text is a dense invisible (Tr 3) OCR layer — the typical output of OCRmyPDF and other re-OCR pipelines — extract zero text and are reported as "OCR is required", even though the exact text is already embedded in the PDF.
## Root cause
`process_document` has an invisible-text retry that recovers Tr 3 OCR layers, but it only runs when the document is classified `Mixed`. A scanned page with a *dense* OCR overlay carries hundreds of text operators, which pushes it past the template-image threshold (`text_operator_count < 50`) so it is classified `TextBased` instead — and for `TextBased` the retry never fires. Normal extraction skips the invisible text, so the result is an empty extraction and `pages_needing_ocr` covering the whole document.
This affects every OCRmyPDF-processed scan I tested (14/14 documents, 1–20 pages each, dense Dutch/German administrative documents): all of them fail with "PDF has no extractable text … OCR is required" while `pdftotext`/pypdf happily read the embedded layer.
## Suggested fix
Widen the invisible retry to `Mixed | TextBased`, and also trigger it when extraction is suspiciously sparse for the page count (I used `items < 5 * page_count`). The existing zero-visible-text gate in the invisible pass already prevents duplicating visible text, so the retry stays behavior-safe for genuine text PDFs: it only fires when normal extraction produced garbage, nothing, or a near-empty result.
With that change all 14 previously-failing documents extract correctly (the densest one, 20 pages / ~376K chars of text layer, in ~0.3 s), and the full test suite still passes.
I have this patch running in production and can open a PR with the fix plus a regression test (a synthetic dense Tr 3 overlay fixture that classifies as `TextBased`).
## Environment
pdf-inspector 1.14.2 (also reproduced on 0.1.7), via anydoc 0.1.9.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start at the process_document entry point and inspect the existing invisible-text retry and the TextBased/Mixed classification path. Reproduce the issue with a dense Tr 3 OCR overlay or the mentioned OCRmyPDF output, then verify that sparse extraction triggers the retry and that the full test suite passes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 50/100