docling-project / docling-project/docling

[Feat] Add "Layout-First" Surgical OCR mode for high-res (72MP+) archival scans

Open
#3,499 2 comments 1 reaction 0 assignees View on GitHub
enhancement ocr
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Problem
When processing high-resolution archival scans (e.g., 72MP newspaper broadsheets from 1856), the standard `docling` Tesseract backend fails with a fatal buffer overflow in Leptonica:
`Error in pixCreateHeader: requested bytes >= 2^31`.

This occurs because the current pipeline attempts to scale and decode the entire 72MP image into a single 32-bit addressable memory buffer. While aggressive downscaling (scale < 1.0) can avoid the crash, it destroys the fidelity of small 19th-century typefaces. Furthermore, even with neural backends (EasyOCR/RapidOCR), multi-column layouts frequently "collapse," merging text across columns and destroying the logical reading order.

### Proposal: "Layout-First" Surgical OCR
We propose a hybrid pipeline that leverages the strengths of the Heron model while respecting the memory constraints of legacy OCR engines:

1. **Layout Phase:** Run Heron at a memory-safe scale (e.g., `images_scale=0.2`) with OCR disabled (`do_ocr=False`) to extract the document schema and bounding boxes.
2. **Surgical Cropping:** Use the high-resolution source image (e.g., via PIL) and mathematically map Heron's low-res coordinates to the raw pixel space.
3. **Targeted OCR:** Iteratively crop each detected layout block and feed these small, manageable strips to Tesseract individually.
4. **Adaptive Pre-processing:** Apply light binarization/thresholding to each crop to clean up paper noise before OCR.
5. **Reassembly:** Stitch the high-fidelity text back into the final Markdown/JSON based on the logical reading order already determined by Heron.

### Benefits
- **Stability:** Enables processing of 100MP+ images on standard hardware without OOM/buffer errors.
- **Fidelity:** Preserves original archival resolution for the recognition stage.
- **Layout Integrity:** Eliminates "column-salad" by isolating blocks before they reach the OCR engine.

### Reference Implementation
We have developed a standalone script demonstrating this technique: [https://gist.github.com/Manamama-Gemini-Cloud-AI-01/f95dca1caf269751769c05a24bbe9d8f](https://gist.github.com/Manamama-Gemini-Cloud-AI-01/f95dca1caf269751769c05a24bbe9d8f)

As the saying goes, *"verba docent, exempla et images trahunt."* In our tests on an 1856 broadsheet reporting the Sumner-Brooks affair, this "Layout-First" approach was the only method that produced human-believable, searchable text without crashing the host system.

We would love to see a native `--layout-first` or `--surgical-ocr` switch in the Docling CLI to support this archival workflow.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.