docling-project / docling-project/docling-core
Feature proposal: per-item OCR confidence on document-level TextItem and TableCell
- Dominant language
- HTML
- Stars
- 282
- Forks
- 214
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 21
Description
**Requested feature**
An optional `confidence: float | None` (default `None`, `ge=0, le=1`, `CONFID_PREC` serializer, matching `BasePrediction`) on document-level `TextItem` and `TableCell`, so consumers can route low-confidence lines to a correction step (cf. docling [#2711](https://github.com/docling-project/docling/issues/2711)) or filter table cells without re-running OCR.
**Current state**
`BasePrediction` carries confidence into several `meta` fields, and the page-level `TableCell` has `confidence: float = 1.0` with `from_ocr`. The document-level items have nothing; docling's `ConfidenceReport` aggregates per page but does not expose per item.
**Why a first-class field rather than `meta`**
`TableData.table_cells` are plain models with no `meta` field, so per-cell confidence has no home there. For `TextItem`, a first-class field is queryable and projector-aware, and can propagate through the chunkers rather than depending on namespaced custom fields.
**Semantics**
Recognition confidence from the producing engine. `None` keeps absence distinguishable from digital-born text (unlike the page-level `1.0` convention, since document-level items mix both sources).
**Versioning**
Happy to ship the full #703 regime with the PR: `CURRENT_VERSION` bump, a registered downgrade projector that strips the fields, schema snapshot and tests.
**Alternatives considered**
- Extending `ProvenanceItem`: rejected, recognition confidence is a property of the item, not of one provenance entry, and multiple `prov` entries would make it ambiguous.
- Keeping aggregation only in docling's `ConfidenceReport`: rejected, per-item routing is exactly what an aggregate cannot express.
Direction agreed, I'll follow with a PR. For context, I'm also working on table structure correctness in docling ([#4088](https://github.com/docling-project/docling/pull/4088)).
Contributor guide
Research direction
Start by tracing BasePrediction, the document-level TextItem and TableCell models, and TableData.table_cells to understand the existing confidence conventions. The work is complete when both items expose the optional confidence field and the agreed #703 versioning regime includes a downgrade projector, schema snapshot, and tests.
Written by the indexing model from the issue text.
Assessment
- Domain
- backend-api-design
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100