docling-project / docling-project/docling
Support tensorrt backend for paddle via rapidocr
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Requested feature
Rapidocr/Paddle has ben slow even on onnx-gpu. Recently paddlev6 and the new version of rapidocr began to support tensorrt and report speed increases https://rapidai.github.io/RapidOCRDocs/latest/blog/2026/07/05/support-pp-ocrv6-tensorrt-engine/#_3
Investigate:
- does docling want to make tensort a first class backend?
- would that make sense also for other models?
In particular consider also the challenges around deployment to figure out if this is worth it:
- deployment GPUs are homogeneous,
- FP16 accuracy is acceptable
- the TensorRT cache survives restarts.
- pre-baking of the compilation
## details
### workaround
Docling currently has no "tensorrt" backend literal. Manually set parameters
```
ocr_options = RapidOcrOptions(
# Docling currently has no "tensorrt" backend literal.
# Using onnxruntime selects/downloads the ONNX model artifacts;
# rapidocr_params below overrides actual execution to TensorRT.
backend="onnxruntime",
# Typical language choice; adjust to your corpus.
lang=["english"],
# PP-OCR document pages are usually upright.
# Leave False unless orientation classification is required.
use_cls=False,
rapidocr_params={
# TensorRT execution
"Det.engine_type": EngineType.TENSORRT,
"Rec.engine_type": EngineType.TENSORRT,
```
Contributor guide
Research direction
Start by tracing RapidOcrOptions and the rapidocr_params entry point shown in the workaround. Review how the current onnxruntime backend is selected and assess the listed deployment constraints, including GPU homogeneity, FP16 accuracy, and TensorRT cache persistence. Done should include a clear decision on first-class TensorRT support and whether it should extend to other models.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- ai, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100