docling-project / docling-project/docling

Lazy-load transformers/torch in pipeline_options_vlm_model / pipeline_options_asr_model so models-onnxruntime-only installs don't import torch at module import

Open
#3,997 7 comments 0 reactions 0 assignees View on GitHub
enhancement triage/close-implemented
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

Summary
-------
Importing docling.datamodel.pipeline_options (or docling.document_converter) on a models-onnxruntime-only install (docling-slim with only ONNX-backed models) currently pulls transformers → torch at module load via pipeline_options_vlm_model / pipeline_options_asr_model. This forces ONNX-only consumers to install torch/transformers, defeating the purpose of the models-onnxruntime extra.

Problem statement
-----------------
- Importing docling.datamodel.pipeline_options (needed for PdfPipelineOptions) or docling.document_converter causes torch to be present in sys.modules even when no torch-backed pipeline stages are used.
- Repro (observed with docling-slim==2.119.0):
import sys
import docling.datamodel.pipeline_options # or import docling.document_converter
assert "torch" not in sys.modules # fails

Root cause (traced)
-------------------
The chain that triggers eager torch import is:
docling/datamodel/pipeline_options_asr_model.py
-> from docling.datamodel.pipeline_options_vlm_model import (...)
docling/datamodel/pipeline_options_vlm_model.py:9
-> from transformers import StoppingCriteria
transformers/generation/stopping_criteria.py:8
-> import torch

This import occurs at module load rather than being deferred to runtime/usage.

Proposal
--------
Apply the lazy-import pattern used in #3837 to the pipeline options modules:
- Defer `from transformers import StoppingCriteria` (and related imports) into the runtime site where it's required, or guard it with try/except ImportError to raise a clear error only when a torch-backed option is instantiated.
- Preserve public datamodel interfaces so consumers that never enable VLM/ASR still import the datamodel without pulling torch/transformers.

Acceptance criteria
-------------------
- With docling-slim[models-onnxruntime] (no torch/transformers), executing:
import sys
import docling.datamodel.pipeline_options
assert "torch" not in sys.modules
succeeds.
- No behavior change for users who enable/instantiate VLM/ASR options.
- Add an import-only smoke test to guard regressions.

Patch hint
----------
Replace module-level `from transformers import StoppingCriteria` with a local import inside the function that needs it, or use try/except ImportError and re-raise with guidance.

Reproduction steps
------------------
1. Create a virtualenv without torch/transformers installed.
2. pip install "docling-slim[format-pdf,models-onnxruntime]==2.119.0" rapidocr
3. python -c "import sys; import docling.datamodel.pipeline_options; print('torch' in sys.modules)"
-> expected False; currently True.

Impact
------
Low-to-moderate code changes; significant benefit for ONNX-only deployments.

Contributor guide

Open the contributing guide

Research direction

Start with docling/datamodel/pipeline_options_asr_model.py and docling/datamodel/pipeline_options_vlm_model.py, then trace the module-level transformers import described in the issue. Run the ONNX-only import reproduction and add the requested import-only smoke test. Done means importing docling.datamodel.pipeline_options without torch or transformers while preserving VLM/ASR behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
developer-experience
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.