docling-project / docling-project/docling

Every conversion fails with PydanticSerializationError: Circular reference detected (pipeline_cache regression)

Open
#4,229 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Bug

Every conversion fails immediately, before any document I/O happens, with:

```
pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON:
PydanticSerializationError: Error calling function ``: ValueError: Circular reference detected (id repeated)
```

This isn't specific to any input file, URL, or pipeline options. It reproduces with a bare default `PdfPipelineOptions()`:

```python
from docling.datamodel.pipeline_options import PdfPipelineOptions
opts = PdfPipelineOptions()
opts.model_dump_json(serialize_as_any=True) # raises PydanticSerializationError
```

Bisected this to #4076 ("fix: avoid pipeline option cache key collisions"), which added `docling/utils/pipeline_cache.py::create_pipeline_options_hash`. That function calls `pipeline_options.model_dump_json(serialize_as_any=True)` to build a cache key, and `document_converter.py::_get_pipeline` calls it on every conversion. With `serialize_as_any=True`, pydantic-core's cycle guard false-positives on shared singleton objects reachable from multiple branches of the default option tree. There's no actual cycle.

The regression spans v2.123.1 (first release containing #4076) through the current latest, v2.126.0. Downgrading to v2.122.0 avoids it. `model_dump_json(serialize_as_any=True)` succeeds there on the same default options object.

### Steps to reproduce

1. `pip install docling==2.126.0`
2. `docling https://arxiv.org/pdf/2206.01062 --output ./out` (or convert any local file, e.g. a plain `.txt`)
3. Observe the traceback below, raised from `_get_pipeline` → `create_pipeline_options_hash` → `model_dump_json(serialize_as_any=True)`, before any document is read or fetched.

Minimal repro without the CLI:

```python
from docling.datamodel.pipeline_options import PdfPipelineOptions
PdfPipelineOptions().model_dump_json(serialize_as_any=True)
```

Full traceback:

```
Traceback (most recent call last):
File "/opt/homebrew/bin/docling", line 6, in
sys.exit(app())
File ".../docling/cli/main.py", line 1667, in convert
export_documents(
File ".../docling/cli/main.py", line 500, in export_documents
for conv_res in conv_results:
File ".../docling/document_converter.py", line 635, in convert_all
for conv_res in conv_res_iter:
File ".../docling/document_converter.py", line 764, in _convert
for item in map(
File ".../docling/document_converter.py", line 811, in _process_document
conv_res = self._execute_pipeline(in_doc, raises_on_error=raises_on_error)
File ".../docling/document_converter.py", line 837, in _execute_pipeline
pipeline = self._get_pipeline(in_doc.format)
File ".../docling/document_converter.py", line 784, in _get_pipeline
options_hash = create_pipeline_options_hash(pipeline_options)
File ".../docling/utils/pipeline_cache.py", line 22, in create_pipeline_options_hash
payload = type(pipeline_options).__qualname__ + pipeline_options.model_dump_json(
File ".../pydantic/main.py", line 519, in model_dump_json
return self.__pydantic_serializer__.to_json(
pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: PydanticSerializationError: Error calling function ``: ValueError: Circular reference detected (id repeated)
```

### Docling version

```
Docling version: 2.126.0
Docling Core version: 2.96.0
Docling IBM Models version: 4.0.2
Docling Parse version: 7.19.1
Python: cpython-311 (3.11.16)
Platform: macOS-26.6.2-arm64-arm-64bit
```

pydantic 2.11.9, pydantic_core 2.33.2

### Python version

```
Python 3.11.16
```

Contributor guide

Open the contributing guide

Research direction

Start with docling/utils/pipeline_cache.py::create_pipeline_options_hash and document_converter.py::_get_pipeline, then reproduce the failure with the minimal PdfPipelineOptions model_dump_json call. Verify that the pipeline options can be serialized for caching and that a CLI or direct document conversion completes without the PydanticSerializationError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.