firecrawl / firecrawl/pdf-inspector
Expose signals the detector already computes: reading direction and per-page confidence
- Dominant language
- Rust
- Stars
- 19.1k
- Forks
- 1.3k
- Avg merge
- 9h 21m
- Merged PRs (30d)
- 51
Description
## Summary
Two signals are computed inside the library and discarded before the public API. Both are cheap to surface, neither changes behaviour, and both would have turned recent bug reports into self-diagnosis.
This follows the same pattern as #36 ("the detector computes `PageAnalysis.has_images` then discards it before the public API").
## 1. Reading direction
`is_rtl_text` already decides whether the rightmost column is read first — it drives `src/extractor/reading_order.rs`. But `is_rtl_char` and `is_rtl_text` are `pub(crate)`, and nothing in `PdfResult`, `PageMarkdownResult` or `TextItem` says which direction was applied.
Grepping the public surface: `src/lib.rs` contains the string `rtl` zero times, and `language` zero times.
**Why it matters:** when RTL handling misfires (#212), the caller sees plausible-looking text and no indication that anything script-specific happened. There is no way to answer "did the RTL path fire on this page?" from the outside, so a silent wrong answer is indistinguishable from a correct one. A `direction: "ltr" | "rtl" | "mixed"` on the page result, or `isRtl` on `TextItem`, makes it self-diagnosing.
It is also useful in its own right: a caller rendering extracted text needs to know the base direction, and today has to re-derive it by inspecting code points.
## 2. Per-page confidence
`PdfClassification` exposes one `confidence: number` for the whole document, alongside a per-page `pagesNeedingOcr` array. So the caller learns *which* pages were flagged but not *how strongly*, and cannot see which pages dragged the document score down.
In #213 the result was `Mixed`, confidence 0.7, 24 of 25 pages flagged — with no way to tell whether those 24 pages were marginal or emphatic, or whether one page was pathological and the rest borderline. Diagnosing it required a full comparison run against poppler. `perPageConfidence: number[]` would have made it a one-line report.
## 3. While you're there: document the `ocrReason` vocabulary
`ocrReason` is typed as a bare `string`, and the vocabulary appears to be four literals — `no_text`, `suspected_garbled_text`, `vector_text`, `phantom_empty_row`. Callers routing on these are string-matching against an undocumented, unstable set. A union type (or a documented list) costs nothing and makes the routing contract explicit. Happy to send that as a small PR if useful.
## Why these three together
The library's product thesis is routing: decide locally whether a document needs an expensive OCR service. Every one of these is about making that decision *auditable* by the caller. Right now a wrong routing decision is invisible until someone compares against another extractor — which is how both #212 and #213 were found.
None of these change any behaviour, so they should be safe to land independently of the fixes they'd help diagnose.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with src/lib.rs and the public PdfResult, PageMarkdownResult, TextItem, and PdfClassification types, then trace direction handling in src/extractor/reading_order.rs. Identify where the computed RTL signal, per-page confidence, and ocrReason values are discarded or exposed, and verify that the public API reports them without changing extraction or classification behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- api, backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100