firecrawl / firecrawl/pdf-inspector

Dense invisible OCR text layers (Tr 3) are classified TextBased and skip the invisible-text fallback — "OCR is required" on OCRmyPDF output

Open
#385 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
19.1k
Forks
1.3k
Avg merge
9h 21m
Merged PRs (30d)
51

Description

## Summary

PDFs whose only text is a dense invisible (Tr 3) OCR layer — the typical output of OCRmyPDF and other re-OCR pipelines — extract zero text and are reported as "OCR is required", even though the exact text is already embedded in the PDF.

## Root cause

`process_document` has an invisible-text retry that recovers Tr 3 OCR layers, but it only runs when the document is classified `Mixed`. A scanned page with a *dense* OCR overlay carries hundreds of text operators, which pushes it past the template-image threshold (`text_operator_count < 50`) so it is classified `TextBased` instead — and for `TextBased` the retry never fires. Normal extraction skips the invisible text, so the result is an empty extraction and `pages_needing_ocr` covering the whole document.

This affects every OCRmyPDF-processed scan I tested (14/14 documents, 1–20 pages each, dense Dutch/German administrative documents): all of them fail with "PDF has no extractable text … OCR is required" while `pdftotext`/pypdf happily read the embedded layer.

## Suggested fix

Widen the invisible retry to `Mixed | TextBased`, and also trigger it when extraction is suspiciously sparse for the page count (I used `items < 5 * page_count`). The existing zero-visible-text gate in the invisible pass already prevents duplicating visible text, so the retry stays behavior-safe for genuine text PDFs: it only fires when normal extraction produced garbage, nothing, or a near-empty result.

With that change all 14 previously-failing documents extract correctly (the densest one, 20 pages / ~376K chars of text layer, in ~0.3 s), and the full test suite still passes.

I have this patch running in production and can open a PR with the fix plus a regression test (a synthetic dense Tr 3 overlay fixture that classifies as `TextBased`).

## Environment

pdf-inspector 1.14.2 (also reproduced on 0.1.7), via anydoc 0.1.9.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at the process_document entry point and inspect the existing invisible-text retry and the TextBased/Mixed classification path. Reproduce the issue with a dense Tr 3 OCR overlay or the mentioned OCRmyPDF output, then verify that sparse extraction triggers the retry and that the full test suite passes.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
50/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.