docling-project / docling-project/docling

[Feature] Pluggable OCR / Document Layout / Reading Order stages with optional dots.ocr integration

Open
#2,128 2 comments 2 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

**User need**
In enterprise documents (scanned, multi-column, table-heavy, Korean/English mixed), Docling’s **OCR**, **layout detection**, and **reading order** quality can vary by document type. We need the ability to **selectively swap or mix engines per stage** so we can use the best component for each document, per customer requirement.

**Requested feature**
Introduce a **pluggable, stage-based pipeline** where the three stages—**`ocr`**, **`layout`**, **`reading_order`**—can be independently configured to use:

* Docling’s native engine (default), or
* An external engine (initial target: **dots.ocr**)

**Scope & requirements**

* **Standardized stage interfaces** for `OCR`, `Layout`, `ReadingOrder` with a stable I/O contract (page coords, text spans, bboxes, order indices, confidences).
* **Normalization/adaptation layer** so external outputs map cleanly to Docling’s canonical structures without losing geometry or metadata.
* **Stage-level controls** for confidence thresholds, timeouts, error handling, and **fallback** (e.g., external < threshold ⇒ revert to Docling).
* **Optional merge policy** (e.g., compare/merge by line/region with confidence-aware reconciliation).
* **Configuration entry points** (CLI / YAML / SDK) to pick the engine per stage and pass engine-specific params—no code changes required by end users.
* **Optional dependency model** (e.g., extras) so external engines are not mandatory installs.

**Target external engine (optional)**

* **dots.ocr**

* HF: [https://huggingface.co/rednote-hilab/dots.ocr](https://huggingface.co/rednote-hilab/dots.ocr)
* Blog: [https://github.com/rednote-hilab/dots.ocr/blob/master/assets/blog.md](https://github.com/rednote-hilab/dots.ocr/blob/master/assets/blog.md)
* Motivation: stronger OCR on mixed-language pages, tables, and multi-column layouts; can be used for OCR only while keeping Docling for layout/reading order, or vice-versa if/when available.

**Expected outcome**

* Per-customer, per-document **stage-wise optimization** (swap only what underperforms).
* **Risk-reduced** improvements via partial replacement and automatic fallback.
* Preserve Docling’s downstream ecosystem (bboxes, structure, post-processing) while allowing best-of-breed components.

### Alternatives

* **Improve Docling’s native models only:** simpler, but slower to cover diverse, domain-specific cases.
* **Replace the entire pipeline with an external stack:** higher migration cost; loses Docling’s ecosystem benefits.
* **Rely solely on pre/post-processing:** helps edge cases but doesn’t fix core engine weaknesses.
* **Call external OCR and feed back only plain text:** loses geometry/structure, harming downstream layout/table workflows.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.