docling-project / docling-project/docling
`torch._C._distributed_c10d` how to solve if e.g. on Termux
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
`docling -v` etc. problem resulting in: `No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package` at see below.
### Solution and cause:
Solved via https://github.com/pytorch/pytorch/issues/152285#issuecomment-3289016639 that is `pip install transformers -U ` if e.g. on Termux, where `apt install python-torch` is missing that DISTRIBUTED compilation flag.
### Ref, full error if not above (downgrade or upgrade of `transformers`)
```
~ $ docling --version
WARNING: linker: Warning: "/data/data/com.termux/files/home/.local/lib/python3.12/site-packages/pypdfium2_raw/libpdfium.so" unused DT entry: unknown processor-specific (type 0x70000001 arg 0x0) (ignoring)
Traceback (most recent call last):
File "/data/data/com.termux/files/usr/bin/docling", line 3, in
from docling.cli.main import app
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/docling/cli/main.py", line 72, in
from docling.document_converter import (
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/docling/document_converter.py", line 54, in
from docling.pipeline.standard_pdf_pipeline import StandardPdfPipeline
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/docling/pipeline/standard_pdf_pipeline.py", line 17, in
from docling.models.code_formula_model import CodeFormulaModel, CodeFormulaModelOptions
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/docling/models/code_formula_model.py", line 17, in
from transformers import AutoModelForImageTextToText, AutoProcessor
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/transformers/__init__.py", line 27, in
from . import dependency_versions_check
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/transformers/dependency_versions_check.py", line 16, in
from .utils.versions import require_version, require_version_core
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/transformers/utils/__init__.py", line 24, in
from .auto_docstring import (
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/transformers/utils/auto_docstring.py", line 30, in
from .generic import ModelOutput
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/transformers/utils/generic.py", line 55, in
from ..model_debugging_utils import model_addition_debugger_context
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/transformers/model_debugging_utils.py", line 29, in
import torch.distributed.tensor
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/torch/distributed/tensor/__init__.py", line 4, in
import torch.distributed.tensor._ops # force import all built-in dtensor ops
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/torch/distributed/tensor/_ops/__init__.py", line 2, in
from ._conv_ops import * # noqa: F403
^^^^^^^^^^^^^^^^^^^^^^^^
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/torch/distributed/tensor/_ops/_conv_ops.py", line 7, in
from torch.distributed.tensor._dtensor_spec import DTensorSpec, TensorMeta
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/torch/distributed/tensor/_dtensor_spec.py", line 6, in
from torch.distributed.tensor.placement_types import (
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/torch/distributed/tensor/placement_types.py", line 8, in
import torch.distributed._functional_collectives as funcol
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/torch/distributed/_functional_collectives.py", line 9, in
import torch.distributed.distributed_c10d as c10d
File "/data/data/com.termux/files/usr/lib/python3.12/site-packages/torch/distributed/distributed_c10d.py", line 23, in
from torch._C._distributed_c10d import (
ModuleNotFoundError: No module named 'torch._C._distributed_c10d'; 'torch._C' is not a package
```
Contributor guide
Assessment
This issue has not been assessed yet.