firecrawl / firecrawl/pdf-inspector

extract_pages_markdown_bytes() flags every page as needs_ocr on a plain text-based PDF, contradicting detect_pdf_bytes()

Open
#252 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
19.2k
Forks
1.3k
Avg merge
9h 21m
Merged PRs (30d)
51

Description

Summary

extract_pages_markdown_bytes() classifies every page as needs_ocr=True with empty
markdown
on a PDF that detect_pdf_bytes() / process_pdf_bytes() classify as
pdf_type="text_based" with pages_needing_ocr=[] — on the exact same input bytes. The two
entry points disagree completely on whether the document needs OCR.

This makes extract_pages_markdown_bytes() unusable for per-page reliability checks right
now: it reports OCR-needed for pages that plainly are not scanned/image-based.

Environment

  • pdf-inspector 0.2.6 (PyPI wheel, manylinux_2_17_x86_64)
  • Python 3.12, Linux x86_64

Minimal repro

# pip install pdf-inspector pymupdf
import fitz
import pdf_inspector

doc = fitz.open()
rect = fitz.Rect(72, 72, 523, 750)
for i in range(3):
    page = doc.new_page()
    page.insert_textbox(rect, f"Page {i + 1}. This is a normal paragraph of body text.")
pdf_bytes = doc.tobytes()
doc.close()

det = pdf_inspector.detect_pdf_bytes(pdf_bytes)
print("detect_pdf_bytes:", det.pdf_type, det.confidence, det.pages_needing_ocr)

per_page = pdf_inspector.extract_pages_markdown_bytes(pdf_bytes)
print("extract_pages_markdown_bytes.pages_needing_ocr:", per_page.pages_needing_ocr)
for p in per_page.pages:
    print(f"  page={p.page} needs_ocr={p.needs_ocr} markdown_len={len(p.markdown)}")

Actual output

detect_pdf_bytes: text_based 0.5 []
extract_pages_markdown_bytes.pages_needing_ocr: [1, 2, 3]
  page=0 needs_ocr=True markdown_len=0
  page=1 needs_ocr=True markdown_len=0
  page=2 needs_ocr=True markdown_len=0

(confidence here is 0.5, not high — but pages_needing_ocr=[] from detect_pdf_bytes()
already says no page needs OCR, directly contradicting extract_pages_markdown_bytes()
flagging all three.)

(Note pages_needing_ocr on the per-page result also appears to be 1-indexed — [1, 2, 3]
for a 3-page doc — while PageMarkdown.page is documented as 0-indexed in the .pyi stub;
worth double-checking that indexing is consistent across the per-page API surface too.)

Expected behavior

extract_pages_markdown_bytes() should agree with detect_pdf_bytes() /
process_pdf_bytes() on whether a given text-based PDF needs OCR — at minimum, it shouldn't
flag every page of a plain multi-page text document as needing OCR with empty Markdown.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the minimal reproduction against extract_pages_markdown_bytes(), then compare its result with detect_pdf_bytes() and process_pdf_bytes() on the same bytes. Trace the per-page extraction and OCR classification entry points; done means a plain text-based multi-page PDF returns Markdown, does not flag every page for OCR, agrees with the other APIs, and has consistent page indexing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, rust
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.