0.2.4's needsOcr gate refuses born-digital PDFs that 0.2.3 converts correctly
- Dominant language
- Rust
- Stars
- 21.5k
- Forks
- 1.3k
- Avg merge
- 42m
- Merged PRs (30d)
- 17
Description
**Versions:** `@firecrawl/anydoc` 0.2.4 (integrity `sha512-rfJxa5L+…P1Og==`) vs 0.2.3; macOS arm64,
native binary `anydoc.darwin-arm64.node` sha256 `97477757…f0d5`. CLI, `--ocr reject` (the default).
**Summary.** Since 0.2.4 (27 Aug 2026), `anydoc` exits 3 (`needs OCR`) with zero output on
born-digital PDFs whose text layer is complete. 0.2.3 converts the same files end to end with
exit 0. The refusal is document-level and all-or-nothing, so one page condemns the whole file.
**Corpus.** 83 PDFs (30 born-digital, 53 scans). 0.2.3: 32 converted, 51 refused
(exit 1). 0.2.4: 19 converted, 64 refused — the 51 scans plus **13 born-digital documents**.
**The 13, with the page 0.2.4 objects to** (all readable by pdftotext / pdf.js):
`census_p60-280_income` 5–7/67 · `crm_analytics` 3, 1365, 1613, 2117–8/2129 · `health` 3–5/1417 ·
`marketing_cloud_next` 3/460 · `netzero_cloud_dev_guide` 1314/1349 · `omnistudio` 3–5, 89/813 ·
`reports_and_dashboards` 3/560 · `rfc9110_http_semantics` 21/194 · `salesforce_industries` 3/26 ·
`salesforce_release_notes` (Winter '27, Prince 15.4) 3/1010 · `salesforce_spring25` 3/1101 ·
`salesforce_summer25` 3/1160 · `tableau` 3/18.
In eleven of them the objected page is the **table of contents** (dot-leader lines). 0.2.3 renders
that page as a normal contents list — 2.43 MB of markdown for the 1,010-page document.
**Expected:** a page with a text layer is converted; if a classifier deems it low-confidence, emit
the text and report `pages_needing_ocr` rather than refusing the document.
**Actual:** exit 3, no output, for the entire document.
**Repro:** `anydoc --ocr reject .pdf` on any of the above (Salesforce release notes are
public downloads). Compare with 0.2.3.
**Impact:** any consumer running local-only (no hosted OCR) loses 13 of the 30 born-digital documents
between 0.2.3 and 0.2.4 for no gain in fidelity.
---
*Edited 3 Sep 2026: corpus split corrected (30 born-digital / 53 scans, was 32/51) and the table-of-contents count corrected upward from ten to eleven. See the comment below for detail; no other figure changed.*
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by reproducing the failure with `anydoc --ocr reject .pdf` and compare the same corpus with 0.2.3. Trace the needs-OCR decision and its page/classifier handling in the CLI conversion path; done means born-digital pages with text layers are emitted while low-confidence pages are reported through `pages_needing_ocr` instead of rejecting the whole document.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- cli, tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100