docling-project / docling-project/docling

CPU-only torch + docling: torchvision CUDA/CPU wheel mismatch causes silent import failure

Open
#3,494 1 comment 0 reactions 0 assignees View on GitHub
bug triage/close-stale
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
3d 4h
Merged PRs (30d)
95

Description

### Bug
When installing `docling` alongside CPU-only `torch` (from the PyTorch CPU index), `docling-ibm-models` pulls `torchvision` from PyPI. PyPI's `torchvision` wheel is the CUDA-flavoured build, which registers operators for CUDA dispatch only. When CPU-only `torch` tries to use these operators at runtime, it fails with `RuntimeError: operator torchvision::nms does not exist`.

The failure is particularly insidious because:
- `pip install` succeeds with no errors or warnings
- `import docling` succeeds (lazy loading)
- The error only surfaces on the first real `DocumentConverter.convert()` call
- The error message gives no indication the root cause is a CUDA/CPU wheel mismatch
- The error often cascades through `transformers` lazy imports and surfaces as completely unrelated errors (e.g., `ModuleNotFoundError: TrainingArguments` or numpy ABI warnings)

**Fix:** install `torchvision` from the same PyTorch CPU index as `torch`, *before* installing `docling`:

```bash
pip install --index-url https://download.pytorch.org/whl/cpu \
torch==2.11.0 torchvision==0.26.0
pip install docling==2.93.0
```

**Suggested improvement:** `docling-ibm-models` could document the CPU-only installation path, or add an `[cpu]` extra that constrains torchvision to the CPU wheel.

### Steps to reproduce
1. Start from a clean Python 3.12 environment (e.g., `python:3.12-slim` Docker image)
2. Install CPU-only torch: `pip install --index-url https://download.pytorch.org/whl/cpu torch==2.11.0`
3. Install docling: `pip install docling==2.93.0`
4. Verify torchvision version: `python -c "import torchvision; print(torchvision.__version__)"` — shows `0.26.0` (no `+cpu` suffix = CUDA build from PyPI)
5. Attempt to parse any PDF:

```python
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.pipeline_options import PdfPipelineOptions
opts = PdfPipelineOptions(do_ocr=False, do_table_structure=True)
conv = DocumentConverter(format_options={"pdf": PdfFormatOption(pipeline_options=opts)})
# Fails on convert() with RuntimeError: operator torchvision::nms does not exist
```

### Docling version
2.93.0

### Python version
3.12 (python:3.12-slim Docker image, Debian Bookworm)

Contributor guide

Open the contributing guide

Research direction

Reproduce the failure in a clean Python 3.12 environment using the documented CPU-index installation commands, then inspect docling-ibm-models dependency metadata and the DocumentConverter.convert() entry point. Compare the CPU-index and PyPI torchvision installations. Done means the CPU-only installation path is documented or constrained and PDF conversion completes without the missing-operator error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
build-system, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.