docling-project / docling-project/docling
Docling fails to extract references from complex documents (61.8% accuracy)
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
Docling consistently fails to extract references from complex clinical documents, achieving only 61.8% accuracy (34 out of 55 expected
references) regardless of output format or extraction method attempted.
Observed Issues:
- References are completely missing from Docling's processed output rather than misclassified
- Missing references don't appear in structured output, converted markdown files, or exported text files
- Content appears to be lost during parsing rather than incorrectly categorized
- Issue is specific to complex clinical documents containing multi-column text, embedded tables, and mixed formatting in reference
sections
Impact: 61.8% accuracy is insufficient for regulatory clinical document processing where near-100% accuracy is required for compliance.
...
### Steps to reproduce
1. Process a complex clinical trial report (CTR) document containing:
- 55+ academic references in a dedicated reference section
- Multiple clinical data tables and statistical sections
- Mixed content layout with multi-column text
2. Use Docling's standard structured extraction to extract references
3. Try multiple output formats (markdown, text, structured JSON)
4. Compare extracted references against a golden standard reference list
5. Observe consistent ~62% accuracy with 21 genuine references completely missing from all output formats
...
### Docling version
Docling version: 2.39.0
Docling Core version: 2.39.0
Docling IBM Models version: 3.6.0
Docling Parse version: 4.1.0
Python: cpython-311 (3.11.4)
Platform: macOS-15.4.1-x86_64-i386-64bit
...
### Python version
Python 3.11.4
[3895 ctr synopsis nn-trials redacted.pdf](https://github.com/user-attachments/files/20981284/3895.ctr.synopsis.nn-trials.redacted.pdf)
...
Contributor guide
Assessment
This issue has not been assessed yet.