firecrawl / firecrawl/pdf-inspector

Expose configurable confidence threshold for OCR routing and document threshold behavior

Open
#267 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
19.1k
Forks
1.3k
Avg merge
9h 21m
Merged PRs (30d)
51

Description

## Description

`pdf-inspector` reports a confidence score and per-page OCR routing, but it is not clear whether callers can adjust the threshold used to decide when a page or document should be routed to OCR. A small configuration option would make the routing behavior easier to tune for different workloads.

## Motivation

In production, the optimal threshold depends on OCR cost, latency, and acceptable extraction quality. Some users may prefer aggressive OCR routing for high recall, while others may prefer local extraction unless confidence is very low. Without an exposed threshold or documentation, users may need to re-implement routing logic or guess the library's default behavior.

## Proposed Solution

- Add an optional configuration field such as `ocr_confidence_threshold` to the classification API in Rust.
- Keep the current default behavior unchanged for backward compatibility.
- Expose the option in Python, Node.js, and WASM bindings where applicable.
- Document the meaning of confidence scores, per-page routing decisions, and how the threshold affects classification.
- Include short examples showing how to route pages based on the returned metadata and configured threshold.

Example Rust API idea:

```rust
let options = ClassificationOptions {
ocr_confidence_threshold: 0.65,
..Default::default()
};
let result = inspect_pdf_with_options(path, options)?;
```

## Alternatives Considered

- Leave the threshold fixed and document only how to interpret the confidence score.
- Require callers to ignore the built-in routing decision and implement custom routing from the confidence score themselves.
- Provide a separate helper function for thresholding instead of adding configuration to the core API.

---
🤖 *Suggested by [GitHub Trend-Hacking Pipeline](https://github.com/yulinlina/start_rise)*

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the Rust classification API and trace how confidence scores and per-page OCR routing are produced. Then inspect the Python, Node.js, and WASM bindings mentioned in the issue, along with the existing documentation and examples. Done means an optional threshold preserves the current default, is exposed where applicable, and its behavior is documented and tested.

Written by the indexing model from the issue text.

Assessment

Tech stack
node.js, python, rust, wasm
Domain
api, backend, documentation
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.