docling-project / docling-project/docling
Every conversion fails with PydanticSerializationError: Circular reference detected (pipeline_cache regression)
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
Every conversion fails immediately, before any document I/O happens, with:
```
pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON:
PydanticSerializationError: Error calling function ``: ValueError: Circular reference detected (id repeated)
```
This isn't specific to any input file, URL, or pipeline options. It reproduces with a bare default `PdfPipelineOptions()`:
```python
from docling.datamodel.pipeline_options import PdfPipelineOptions
opts = PdfPipelineOptions()
opts.model_dump_json(serialize_as_any=True) # raises PydanticSerializationError
```
Bisected this to #4076 ("fix: avoid pipeline option cache key collisions"), which added `docling/utils/pipeline_cache.py::create_pipeline_options_hash`. That function calls `pipeline_options.model_dump_json(serialize_as_any=True)` to build a cache key, and `document_converter.py::_get_pipeline` calls it on every conversion. With `serialize_as_any=True`, pydantic-core's cycle guard false-positives on shared singleton objects reachable from multiple branches of the default option tree. There's no actual cycle.
The regression spans v2.123.1 (first release containing #4076) through the current latest, v2.126.0. Downgrading to v2.122.0 avoids it. `model_dump_json(serialize_as_any=True)` succeeds there on the same default options object.
### Steps to reproduce
1. `pip install docling==2.126.0`
2. `docling https://arxiv.org/pdf/2206.01062 --output ./out` (or convert any local file, e.g. a plain `.txt`)
3. Observe the traceback below, raised from `_get_pipeline` → `create_pipeline_options_hash` → `model_dump_json(serialize_as_any=True)`, before any document is read or fetched.
Minimal repro without the CLI:
```python
from docling.datamodel.pipeline_options import PdfPipelineOptions
PdfPipelineOptions().model_dump_json(serialize_as_any=True)
```
Full traceback:
```
Traceback (most recent call last):
File "/opt/homebrew/bin/docling", line 6, in
sys.exit(app())
File ".../docling/cli/main.py", line 1667, in convert
export_documents(
File ".../docling/cli/main.py", line 500, in export_documents
for conv_res in conv_results:
File ".../docling/document_converter.py", line 635, in convert_all
for conv_res in conv_res_iter:
File ".../docling/document_converter.py", line 764, in _convert
for item in map(
File ".../docling/document_converter.py", line 811, in _process_document
conv_res = self._execute_pipeline(in_doc, raises_on_error=raises_on_error)
File ".../docling/document_converter.py", line 837, in _execute_pipeline
pipeline = self._get_pipeline(in_doc.format)
File ".../docling/document_converter.py", line 784, in _get_pipeline
options_hash = create_pipeline_options_hash(pipeline_options)
File ".../docling/utils/pipeline_cache.py", line 22, in create_pipeline_options_hash
payload = type(pipeline_options).__qualname__ + pipeline_options.model_dump_json(
File ".../pydantic/main.py", line 519, in model_dump_json
return self.__pydantic_serializer__.to_json(
pydantic_core._pydantic_core.PydanticSerializationError: Error serializing to JSON: PydanticSerializationError: Error calling function ``: ValueError: Circular reference detected (id repeated)
```
### Docling version
```
Docling version: 2.126.0
Docling Core version: 2.96.0
Docling IBM Models version: 4.0.2
Docling Parse version: 7.19.1
Python: cpython-311 (3.11.16)
Platform: macOS-26.6.2-arm64-arm-64bit
```
pydantic 2.11.9, pydantic_core 2.33.2
### Python version
```
Python 3.11.16
```
Contributor guide
Research direction
Start with docling/utils/pipeline_cache.py::create_pipeline_options_hash and document_converter.py::_get_pipeline, then reproduce the failure with the minimal PdfPipelineOptions model_dump_json call. Verify that the pipeline options can be serialized for caching and that a CLI or direct document conversion completes without the PydanticSerializationError.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100