docling-project / docling-project/docling

Inconsistent layout detection on a publicly available PDF corpus

Open
#2,650 3 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
3d 4h
Merged PRs (30d)
98

Description

### Bug
Hi, I work at [Demokratis](https://github.com/Demokratis-ch/demokratis-ml) where we process a lot of PDFs containing, among others, drafts of laws being proposed for consultations. When running Docling on these drafts the layout is usually detected inconsistently or downright incorrectly. The main issues are:

- Headings are often not recognised as such and are extracted as paragraphs.
- List items are sometimes also extracted as paragraphs. When they're correctly identified as list items, their numbering (e.g. "1)" or "1bis" is not often a part of the text instead of being separated as "list-style-type".
- Footnotes flow with the main text and are not identified as footnotes in any way.
- Superscript (most often references to footnotes) are not recognised and are part of the normal text.

### Steps to reproduce
Using this sample code you can access the dataset, filter it as desired, and compare the original PDF with the extracted structure. There are about 3k documents available in each of German, French, and Italian.

```python
import pandas as pd
from docling.document_converter import DocumentConverter

df = pd.read_parquet(
# This is a dataset of all kinds of documents pertaining to Swiss consultation procedures (Vernehmlassungsverfahren).
"https://huggingface.co/datasets/demokratis/consultation-documents/resolve/main/consultation-documents-preprocessed.parquet",
# Restrict the columns so that we don't have to download the entire 2 GB dataframe.
columns=["consultation_start_date", "political_body", "document_type", "document_language", "document_source_url"],
)
df = df.loc[
(df["document_language"] == "de") # "fr" and "it" are also available
& (df["political_body"] == "ch") # filter for federal documents
& (df["document_type"] == "DRAFT") # filter for legal drafts - they have a very regular structure
& (df["consultation_start_date"].dt.year >= 2010) # look at recent documents only (optional)
]

document_url = df["document_source_url"].sample().iloc[0]
print("-" * 50)
print(document_url)
print("-" * 50)

converter = DocumentConverter()
result = converter.convert(document_url)
print("-" * 50)
print(result.document.export_to_html())
```

### Docling version
Docling version: 2.62.0
Docling Core version: 2.51.1
Docling IBM Models version: 3.10.2
Docling Parse version: 4.7.1
Python: cpython-312 (3.12.1)
Platform: macOS-14.8-arm64-arm-64bit

### Python version
Python 3.12.1

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.