firecrawl / firecrawl/pdf-inspector
extract_pages_markdown_bytes() flags every page as needs_ocr on a plain text-based PDF, contradicting detect_pdf_bytes()
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 19.2k
- Forks
- 1.3k
- Avg merge
- 9h 21m
- Merged PRs (30d)
- 51
Description
Summary
extract_pages_markdown_bytes() classifies every page as needs_ocr=True with empty
markdown on a PDF that detect_pdf_bytes() / process_pdf_bytes() classify as
pdf_type="text_based" with pages_needing_ocr=[] — on the exact same input bytes. The two
entry points disagree completely on whether the document needs OCR.
This makes extract_pages_markdown_bytes() unusable for per-page reliability checks right
now: it reports OCR-needed for pages that plainly are not scanned/image-based.
Environment
pdf-inspector0.2.6 (PyPI wheel,manylinux_2_17_x86_64)- Python 3.12, Linux x86_64
Minimal repro
# pip install pdf-inspector pymupdf
import fitz
import pdf_inspector
doc = fitz.open()
rect = fitz.Rect(72, 72, 523, 750)
for i in range(3):
page = doc.new_page()
page.insert_textbox(rect, f"Page {i + 1}. This is a normal paragraph of body text.")
pdf_bytes = doc.tobytes()
doc.close()
det = pdf_inspector.detect_pdf_bytes(pdf_bytes)
print("detect_pdf_bytes:", det.pdf_type, det.confidence, det.pages_needing_ocr)
per_page = pdf_inspector.extract_pages_markdown_bytes(pdf_bytes)
print("extract_pages_markdown_bytes.pages_needing_ocr:", per_page.pages_needing_ocr)
for p in per_page.pages:
print(f" page={p.page} needs_ocr={p.needs_ocr} markdown_len={len(p.markdown)}")
Actual output
detect_pdf_bytes: text_based 0.5 []
extract_pages_markdown_bytes.pages_needing_ocr: [1, 2, 3]
page=0 needs_ocr=True markdown_len=0
page=1 needs_ocr=True markdown_len=0
page=2 needs_ocr=True markdown_len=0
(confidence here is 0.5, not high — but pages_needing_ocr=[] from detect_pdf_bytes()
already says no page needs OCR, directly contradicting extract_pages_markdown_bytes()
flagging all three.)
(Note pages_needing_ocr on the per-page result also appears to be 1-indexed — [1, 2, 3]
for a 3-page doc — while PageMarkdown.page is documented as 0-indexed in the .pyi stub;
worth double-checking that indexing is consistent across the per-page API surface too.)
Expected behavior
extract_pages_markdown_bytes() should agree with detect_pdf_bytes() /
process_pdf_bytes() on whether a given text-based PDF needs OCR — at minimum, it shouldn't
flag every page of a plain multi-page text document as needing OCR with empty Markdown.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running the minimal reproduction against extract_pages_markdown_bytes(), then compare its result with detect_pdf_bytes() and process_pdf_bytes() on the same bytes. Trace the per-page extraction and OCR classification entry points; done means a plain text-based multi-page PDF returns Markdown, does not flag every page for OCR, agrees with the other APIs, and has consistent page indexing.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, rust
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100