docling-project / docling-project/docling
Lazy-load transformers/torch in pipeline_options_vlm_model / pipeline_options_asr_model so models-onnxruntime-only installs don't import torch at module import
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
Summary
-------
Importing docling.datamodel.pipeline_options (or docling.document_converter) on a models-onnxruntime-only install (docling-slim with only ONNX-backed models) currently pulls transformers → torch at module load via pipeline_options_vlm_model / pipeline_options_asr_model. This forces ONNX-only consumers to install torch/transformers, defeating the purpose of the models-onnxruntime extra.
Problem statement
-----------------
- Importing docling.datamodel.pipeline_options (needed for PdfPipelineOptions) or docling.document_converter causes torch to be present in sys.modules even when no torch-backed pipeline stages are used.
- Repro (observed with docling-slim==2.119.0):
import sys
import docling.datamodel.pipeline_options # or import docling.document_converter
assert "torch" not in sys.modules # fails
Root cause (traced)
-------------------
The chain that triggers eager torch import is:
docling/datamodel/pipeline_options_asr_model.py
-> from docling.datamodel.pipeline_options_vlm_model import (...)
docling/datamodel/pipeline_options_vlm_model.py:9
-> from transformers import StoppingCriteria
transformers/generation/stopping_criteria.py:8
-> import torch
This import occurs at module load rather than being deferred to runtime/usage.
Proposal
--------
Apply the lazy-import pattern used in #3837 to the pipeline options modules:
- Defer `from transformers import StoppingCriteria` (and related imports) into the runtime site where it's required, or guard it with try/except ImportError to raise a clear error only when a torch-backed option is instantiated.
- Preserve public datamodel interfaces so consumers that never enable VLM/ASR still import the datamodel without pulling torch/transformers.
Acceptance criteria
-------------------
- With docling-slim[models-onnxruntime] (no torch/transformers), executing:
import sys
import docling.datamodel.pipeline_options
assert "torch" not in sys.modules
succeeds.
- No behavior change for users who enable/instantiate VLM/ASR options.
- Add an import-only smoke test to guard regressions.
Patch hint
----------
Replace module-level `from transformers import StoppingCriteria` with a local import inside the function that needs it, or use try/except ImportError and re-raise with guidance.
Reproduction steps
------------------
1. Create a virtualenv without torch/transformers installed.
2. pip install "docling-slim[format-pdf,models-onnxruntime]==2.119.0" rapidocr
3. python -c "import sys; import docling.datamodel.pipeline_options; print('torch' in sys.modules)"
-> expected False; currently True.
Impact
------
Low-to-moderate code changes; significant benefit for ONNX-only deployments.
Contributor guide
Research direction
Start with docling/datamodel/pipeline_options_asr_model.py and docling/datamodel/pipeline_options_vlm_model.py, then trace the module-level transformers import described in the issue. Run the ONNX-only import reproduction and add the requested import-only smoke test. Done means importing docling.datamodel.pipeline_options without torch or transformers while preserving VLM/ASR behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- developer-experience
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100