docling-project / docling-project/docling

Bug: VLM-generated Markdown with bracketed LaTeX causes crash in document assembly

Open
#1,950 7 comments 2 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
3d 4h
Merged PRs (30d)
98

Description

### Bug

Hello, I'm sharing Gemini diagnostic, but not entirely sure that is relevant:

The VlmPipeline fails during the document assembly stage when processing Markdown generated from a PDF that contains a multi-line LaTeX array environment. The VLM model (mistral-community/pixtral-12b-240910) sometimes wraps complex LaTeX blocks in square brackets ([...]) instead of the standard ... delimiters.The internal docling Markdown parser then incorrectly interprets this block as a single element with nested children (, , etc.). This complex, nested structure is passed to document.append_child_item(), which expects a "flat" element, triggering a ValueError: Can not append a child with children.The error originates in docling/pipeline/vlm_pipeline.py during the _turn_md_into_doc step, which attempts to build the document from the faulty Markdown.

### Steps to reproduce

```python
import logging
from pathlib import Path

from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import (
VlmPipelineOptions,
)
from docling.datamodel.pipeline_options_vlm_model import ApiVlmOptions, ResponseFormat
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline

def lms_vlm_options(model: str, prompt: str, format: ResponseFormat):
options = ApiVlmOptions(
url="http://localhost:8000/v1/chat/completions",
api_key="",
params=dict(
model=model,
),
prompt=prompt,
timeout=90,
max_size=16024,
temperature=0.0,
response_format=format,
)
return options

def main():
logging.basicConfig(level=logging.DEBUG)

# Download the file from: http://www.doctortang.com/Algebra%202/Essential%20Formulas%20for%20Algebra%202%20Final%20Exam.pdf
# And save it as "algebra_formulas.pdf"
input_doc_path = Path("algebra_formulas.pdf")

pipeline_options = VlmPipelineOptions(
enable_remote_services=True
)

pipeline_options.vlm_options = lms_vlm_options(
model="mistral-community/pixtral-12b-240910",
prompt="OCR the full page to markdown.",
format=ResponseFormat.MARKDOWN,
)

doc_converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_options=pipeline_options,
pipeline_cls=VlmPipeline,
)
}
)

try:
result = doc_converter.convert(input_doc_path)
print(result.document.export_to_markdown())
except ValueError as e:
print(f"Caught expected error: {e}")

if __name__ == "__main__":
main()
```

### Logs

The script will fail with the following traceback when processing page 4 of the PDF:DEBUG:docling.backend.md_backend: Some other element:
```bash
,
,
,
,
,
,
,
,
,
,
]>
...
Traceback (most recent call last):
File "/demo/docling/demo.py", line 69, in
main()
~~~~^^
File "/demo/docling/demo.py", line 66, in main
result = doc_converter.convert(input_doc_path)
...
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling/pipeline/base_pipeline.py", line 47, in execute
conv_res = self._assemble_document(conv_res)
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling/pipeline/vlm_pipeline.py", line 147, in _assemble_document
conv_res.document = self._turn_md_into_doc(conv_res)
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling/pipeline/vlm_pipeline.py", line 300, in _turn_md_into_doc
conv_res.document.append_child_item(child=item)
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling_core/types/doc/document.py", line 1956, in append_child_item
raise ValueError("Can not append a child with children")
ValueError: Can not append a child with children
```

### Docling version

Docling version: 2.41.0
Docling Core version: 2.42.0
Docling IBM Models version: 3.8.1
Docling Parse version: 4.1.0
Python: cpython-313 (3.13.5)
Platform: Linux-5.15.0-143-generic-x86_64-with-glibc2.35

### Python version
Python 3.13.5

Thank you for your help! :)

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the failure with the provided PDF and VLM configuration, then inspect docling/pipeline/vlm_pipeline.py, especially _turn_md_into_doc and the append_child_item call. Done means the bracketed multi-line LaTeX Markdown no longer raises “Can not append a child with children” during document assembly and the conversion completes.

Written by the indexing model from the issue text.

Assessment

Tech stack
markdown, python
Domain
ai, backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
68/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.