docling-project / docling-project/docling
CPU-only torch + docling: torchvision CUDA/CPU wheel mismatch causes silent import failure
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 95
Description
### Bug
When installing `docling` alongside CPU-only `torch` (from the PyTorch CPU index), `docling-ibm-models` pulls `torchvision` from PyPI. PyPI's `torchvision` wheel is the CUDA-flavoured build, which registers operators for CUDA dispatch only. When CPU-only `torch` tries to use these operators at runtime, it fails with `RuntimeError: operator torchvision::nms does not exist`.
The failure is particularly insidious because:
- `pip install` succeeds with no errors or warnings
- `import docling` succeeds (lazy loading)
- The error only surfaces on the first real `DocumentConverter.convert()` call
- The error message gives no indication the root cause is a CUDA/CPU wheel mismatch
- The error often cascades through `transformers` lazy imports and surfaces as completely unrelated errors (e.g., `ModuleNotFoundError: TrainingArguments` or numpy ABI warnings)
**Fix:** install `torchvision` from the same PyTorch CPU index as `torch`, *before* installing `docling`:
```bash
pip install --index-url https://download.pytorch.org/whl/cpu \
torch==2.11.0 torchvision==0.26.0
pip install docling==2.93.0
```
**Suggested improvement:** `docling-ibm-models` could document the CPU-only installation path, or add an `[cpu]` extra that constrains torchvision to the CPU wheel.
### Steps to reproduce
1. Start from a clean Python 3.12 environment (e.g., `python:3.12-slim` Docker image)
2. Install CPU-only torch: `pip install --index-url https://download.pytorch.org/whl/cpu torch==2.11.0`
3. Install docling: `pip install docling==2.93.0`
4. Verify torchvision version: `python -c "import torchvision; print(torchvision.__version__)"` — shows `0.26.0` (no `+cpu` suffix = CUDA build from PyPI)
5. Attempt to parse any PDF:
```python
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipeline_options import PdfPipelineOptions
opts = PdfPipelineOptions(do_ocr=False, do_table_structure=True)
conv = DocumentConverter(format_options={"pdf": PdfFormatOption(pipeline_options=opts)})
# Fails on convert() with RuntimeError: operator torchvision::nms does not exist
```
### Docling version
2.93.0
### Python version
3.12 (python:3.12-slim Docker image, Debian Bookworm)
Contributor guide
Research direction
Reproduce the failure in a clean Python 3.12 environment using the documented CPU-index installation commands, then inspect docling-ibm-models dependency metadata and the DocumentConverter.convert() entry point. Compare the CPU-index and PyPI torchvision installations. Done means the CPU-only installation path is documented or constrained and PDF conversion completes without the missing-operator error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- build-system, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100