docling-project / docling-project/docling

Stabilize PyPdfium2 backend and LayoutPostprocessor for standalone ONNX inference pipelines

Open
#3,996 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
3d 4h
Merged PRs (30d)
95

Description

Summary
-------
I’m targeting a torch-free, PyMuPDF-free, ONNX-only deployment where I run my own ONNX inference loop.
To integrate rendering, text extraction, and postprocessing, I’d like to confirm whether parts of
docling are intended to be usable standalone and suggest minor API/documentation work to make that integration straightforward.

Scope
-----
1) LayoutPostprocessor (docling/utils/layout_postprocessor.py)
- Implements document-aware clustering/dedup described in the technical report.
- Has no torch dependency (only rtree + docling_core types).
- Questions:
- Is it intended to accept raw ONNX detections + custom text cells as a public API?
- If so, can the input surface be documented or adapted (e.g., a small helper function: process(detections, text_cells, page_meta))?
- Note: #2143 documents a no-op in _handle_cross_type_overlaps; that should be considered for safe standalone use.

2) PyPdfium2 backend (docling/backend/pypdfium2_backend.py)
- Is PyPdfiumDocumentBackend intended for standalone use to render pages and extract text-cells (without DocumentConverter)?
- If not currently stable, would it be feasible to expose a small, dependency-minimal API to:
- Render pages to images at controlled DPI/resolution
- Extract text-cells with coordinates and basic metadata
to feed into a separate ONNX inference loop and the LayoutPostprocessor?

Proposal
--------
- Clarify the intended integration surface for each component in the docs (API examples).
- If small API adjustments are acceptable, add thin helpers to accept and return minimal well-documented
data structures (e.g., Detection[], TextCell[], PageMeta) so users can plug their own ONNX loops.
- Optionally add a small example script demonstrating:
pypdfium2_backend → render + extract cells → run ONNX layout model → LayoutPostprocessor

Acceptance criteria
-------------------
- Public docs/examples show how to render/extract text-cells with PyPdfium2 backend and how to call
LayoutPostprocessor with raw detections/text-cells.
- Minimal API changes only; no heavy refactor of pipeline internals.
- Tests or example scripts demonstrating the standalone flow.

Impact
------
- Enables memory- and image-size-constrained, CPU-only ONNX deployments that avoid torch/PyMuPDF.
- Low-to-moderate code changes (helpers + docs + small example).

Related context
---------------
- #2143 — LayoutPostprocessor._handle_cross_type_overlaps seems to be a no-op (needs review).
- This is complementary to the ONNX-only import work in the other feature request.

Contributor guide

Open the contributing guide

Research direction

Start with docling/utils/layout_postprocessor.py and docling/backend/pypdfium2_backend.py, then review related issue #2143. Determine the intended standalone inputs and outputs from the existing components before defining minimal docs or helpers. Done means the standalone rendering, text-cell extraction, and LayoutPostprocessor flow is documented with tests or an example script.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend-api-design, documentation
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.