docling-project / docling-project/docling

torch.compile() may fail in some configuration

Open
#3,986 1 comment 1 reaction 0 assignees View on GitHub
bug triage/close-fixed
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Bug

Docling fails to convert a PDF on Apple Silicon macOS when Torch model compilation is enabled by default.

The conversion reaches the layout-detection stage, then fails inside PyTorch TorchInductor with:

```text
torch._inductor.exc.InductorError:
TypeError: BooleanAtom not allowed in this context.
```

Relevant traceback path:

```text
Docling layout model
-> Transformers object-detection engine
-> torch.compile()
-> TorchDynamo/TorchInductor
-> SymPy symbolic shape handling
-> BooleanAtom not allowed in this context
```

Disabling Docling’s Torch compilation allows the same PDF to convert successfully:

```bash
DOCLING_INFERENCE_COMPILE_TORCH_MODELS=false \
uvx --native-tls docling \
--to md \
-v \
--image-export-mode referenced \
.pdf
```

The workaround produced valid Markdown output and referenced image artifacts. Therefore, the failure appears limited to the compiled inference path rather than PDF parsing or the conversion pipeline itself.

It would be useful if Docling either:

- Automatically fell back to eager inference when torch.compile()/TorchInductor fails; or
- Disabled model compilation by default for unsupported or unstable platform/backend combinations.

### Steps to reproduce

1. Use an Apple Silicon Mac.
2. Obtain a PDF with the relevant characteristics. The original test file was a 64-page encrypted PDF.
3. Run:
```bash
uvx --native-tls docling \
--to md \
-v \
--image-export-mode referenced \
.pdf
```
4. Observe that Docling selects the MPS accelerator:
```text
Accelerator device: 'mps'
```
5. Observe that it initializes the Transformers object-detection layout engine.
6. Observe conversion failure during layout processing:
```text
Stage layout failed
TypeError: BooleanAtom not allowed in this context.
```
7. Run the same conversion with model compilation disabled:
```bash
DOCLING_INFERENCE_COMPILE_TORCH_MODELS=false \
uvx --native-tls docling \
--to md \
-v \
--image-export-mode referenced \
pdf
```
8. Observe that conversion succeeds.

### Docling version

```text
Docling version: 2.119.0
Docling Core version: 2.91.0
Docling IBM Models version: 3.14.0
Docling Parse version: 7.12.1
Python: cpython-312 (3.12.11)
Platform: macOS-26.5.2-arm64-arm-64bit
```

### Python version

```text
Python 3.12.11
```

Additional environment information:

```text
PyTorch: 2.13.0
Device selected by Docling: mps
OS: macOS arm64
```

The exact error is:

```text
torch._inductor.exc.InductorError: TypeError: BooleanAtom not allowed in this context.
```

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with the documented uvx command on Apple Silicon, then compare it with DOCLING_INFERENCE_COMPILE_TORCH_MODELS=false. Trace the Docling layout model's Transformers object-detection engine through torch.compile() and determine how the BooleanAtom error is handled. Done means the affected conversion no longer fails in the compiled path, or reliably falls back to eager inference.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, python, pytorch
Domain
ai, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.