docling-project / docling-project/docling
Bug: VLM-generated Markdown with bracketed LaTeX causes crash in document assembly
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 98
Description
### Bug
Hello, I'm sharing Gemini diagnostic, but not entirely sure that is relevant:
The VlmPipeline fails during the document assembly stage when processing Markdown generated from a PDF that contains a multi-line LaTeX array environment. The VLM model (mistral-community/pixtral-12b-240910) sometimes wraps complex LaTeX blocks in square brackets ([...]) instead of the standard ... delimiters.The internal docling Markdown parser then incorrectly interprets this block as a single element with nested children (, , etc.). This complex, nested structure is passed to document.append_child_item(), which expects a "flat" element, triggering a ValueError: Can not append a child with children.The error originates in docling/pipeline/vlm_pipeline.py during the _turn_md_into_doc step, which attempts to build the document from the faulty Markdown.
### Steps to reproduce
```python
import logging
from pathlib import Path
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import (
VlmPipelineOptions,
)
from docling.datamodel.pipeline_options_vlm_model import ApiVlmOptions, ResponseFormat
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.pipeline.vlm_pipeline import VlmPipeline
def lms_vlm_options(model: str, prompt: str, format: ResponseFormat):
options = ApiVlmOptions(
url="http://localhost:8000/v1/chat/completions",
api_key="",
params=dict(
model=model,
),
prompt=prompt,
timeout=90,
max_size=16024,
temperature=0.0,
response_format=format,
)
return options
def main():
logging.basicConfig(level=logging.DEBUG)
# Download the file from: http://www.doctortang.com/Algebra%202/Essential%20Formulas%20for%20Algebra%202%20Final%20Exam.pdf
# And save it as "algebra_formulas.pdf"
input_doc_path = Path("algebra_formulas.pdf")
pipeline_options = VlmPipelineOptions(
enable_remote_services=True
)
pipeline_options.vlm_options = lms_vlm_options(
model="mistral-community/pixtral-12b-240910",
prompt="OCR the full page to markdown.",
format=ResponseFormat.MARKDOWN,
)
doc_converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_options=pipeline_options,
pipeline_cls=VlmPipeline,
)
}
)
try:
result = doc_converter.convert(input_doc_path)
print(result.document.export_to_markdown())
except ValueError as e:
print(f"Caught expected error: {e}")
if __name__ == "__main__":
main()
```
### Logs
The script will fail with the following traceback when processing page 4 of the PDF:DEBUG:docling.backend.md_backend: Some other element:
```bash
,
,
,
,
,
,
,
,
,
,
]>
...
Traceback (most recent call last):
File "/demo/docling/demo.py", line 69, in
main()
~~~~^^
File "/demo/docling/demo.py", line 66, in main
result = doc_converter.convert(input_doc_path)
...
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling/pipeline/base_pipeline.py", line 47, in execute
conv_res = self._assemble_document(conv_res)
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling/pipeline/vlm_pipeline.py", line 147, in _assemble_document
conv_res.document = self._turn_md_into_doc(conv_res)
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling/pipeline/vlm_pipeline.py", line 300, in _turn_md_into_doc
conv_res.document.append_child_item(child=item)
File "/software/miniconda3/envs/docling_demo/lib/python3.13/site-packages/docling_core/types/doc/document.py", line 1956, in append_child_item
raise ValueError("Can not append a child with children")
ValueError: Can not append a child with children
```
### Docling version
Docling version: 2.41.0
Docling Core version: 2.42.0
Docling IBM Models version: 3.8.1
Docling Parse version: 4.1.0
Python: cpython-313 (3.13.5)
Platform: Linux-5.15.0-143-generic-x86_64-with-glibc2.35
### Python version
Python 3.13.5
Thank you for your help! :)
Contributor guide
Research direction
Start by reproducing the failure with the provided PDF and VLM configuration, then inspect docling/pipeline/vlm_pipeline.py, especially _turn_md_into_doc and the append_child_item call. Done means the bracketed multi-line LaTeX Markdown no longer raises “Can not append a child with children” during document assembly and the conversion completes.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- markdown, python
- Domain
- ai, backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100