docling-project / docling-project/docling

Support tensorrt backend for paddle via rapidocr

Open
#3,862 0 comments 0 reactions 0 assignees View on GitHub
accelerators dependency mgmt enhancement needs investigation performance
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Requested feature

Rapidocr/Paddle has ben slow even on onnx-gpu. Recently paddlev6 and the new version of rapidocr began to support tensorrt and report speed increases https://rapidai.github.io/RapidOCRDocs/latest/blog/2026/07/05/support-pp-ocrv6-tensorrt-engine/#_3

Investigate:

- does docling want to make tensort a first class backend?
- would that make sense also for other models?

In particular consider also the challenges around deployment to figure out if this is worth it:

- deployment GPUs are homogeneous,
- FP16 accuracy is acceptable
- the TensorRT cache survives restarts.
- pre-baking of the compilation

## details

### workaround

Docling currently has no "tensorrt" backend literal. Manually set parameters

```
ocr_options = RapidOcrOptions(
# Docling currently has no "tensorrt" backend literal.
# Using onnxruntime selects/downloads the ONNX model artifacts;
# rapidocr_params below overrides actual execution to TensorRT.
backend="onnxruntime",

# Typical language choice; adjust to your corpus.
lang=["english"],

# PP-OCR document pages are usually upright.
# Leave False unless orientation classification is required.
use_cls=False,

rapidocr_params={
# TensorRT execution
"Det.engine_type": EngineType.TENSORRT,
"Rec.engine_type": EngineType.TENSORRT,

```

Contributor guide

Open the contributing guide

Research direction

Start by tracing RapidOcrOptions and the rapidocr_params entry point shown in the workaround. Review how the current onnxruntime backend is selected and assess the listed deployment constraints, including GPU homogeneity, FP16 accuracy, and TensorRT cache persistence. Done should include a clear decision on first-class TensorRT support and whether it should extend to other models.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, machine-learning
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.