docling-project / docling-project/docling

PDF: wrapped section_header titles split across two elements — SECTION_HEADER missing from predict_merges (companion to #3881)

Open
#4,016 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Bug

`predict_merges` (docling-ibm-models' `reading_order_rb.py`) reassembles a line-wrapped word split across two clusters, but only for `TEXT` — and, as of #3888 / docling-ibm-models#171, `LIST_ITEM`. `SECTION_HEADER` is not in that filter, so a heading whose title wraps across a line break is left split into two separate `section_header` elements instead of being merged back into one.

Example from a real PDF (Seattle Municipal Code, digitally-native, single-column):

```
3.32.010 Seattle Public Utilities—Gen-
eral Manager and Chief Executive Officer
```

Docling emits these as two `SECTION_HEADER` elements — `"3.32.010 Seattle Public Utilities—Gen-"` and `"eral Manager and Chief Executive Officer"` — rather than one merged `"3.32.010 Seattle Public Utilities—General Manager and Chief Executive Officer"`.

A second example from the same document, this time without a hyphen (the wrap fell at a word boundary, not mid-word):

```
3.30.250 Functional review of board
operations—Abolition or continuation of Board.
```

Again two separate `SECTION_HEADER` elements, rather than one merged heading.

### Steps to reproduce

```python
from docling.document_converter import DocumentConverter

converter = DocumentConverter()
result = converter.convert("seattle_municipal_code.pdf")
for item, _ in result.document.iterate_items():
if getattr(item, "label", None) == "section_header":
print(repr(item.text))

# Emits two separate section_header items instead of one merged heading:
# '3.32.010 Seattle Public Utilities—Gen-'
# 'eral Manager and Chief Executive Officer'
```

### Expected behavior

The wrapped title should be detected and merged into a single `section_header` element, the same way #3888 / docling-ibm-models#171 now do for `LIST_ITEM`: a fragment ending in a hard or soft hyphen followed by a continuation starting with a lowercase letter gets dehyphenated and merged; a fragment with no trailing hyphen followed by a lowercase-starting continuation gets merged with a single space.

### Source location

This looks like the same gap #3881 described for `LIST_ITEM`, one label short of complete: `docling_ibm_models/reading_order/reading_order_rb.py`'s cross-cluster continuation detection in `predict_merges` was extended by PR docling-ibm-models#171 to cover `LIST_ITEM` in addition to `TEXT`, with the companion final-dehyphenation change landing in docling#3888 (merged). `SECTION_HEADER` was not included in either change, and neither issue nor any other open issue/PR in either repo appears to request it.

### Suggestion

Extend the same label filter one more step to include `SECTION_HEADER`, using the identical hard/soft-hyphen + lowercase-continuation heuristic already reviewed and merged for `LIST_ITEM` in docling-ibm-models#171.

### Docling version

2.120.1 (also checked 2.120.2 — gap still present; docling-ibm-models#171 containing the underlying merge-mapping change is still open/unmerged as of this report)

### Related issues/PRs

- #3881 (the original `LIST_ITEM` report)
- #3888 (merged — dehyphenation-on-merge for `LIST_ITEM`/`TEXT`)
- docling-ibm-models#171 (open — the companion cross-cluster mapping change for `LIST_ITEM`)

Contributor guide

Open the contributing guide

Research direction

Start with docling_ibm_models/reading_order/reading_order_rb.py and inspect predict_merges, comparing its SECTION_HEADER handling with the TEXT and LIST_ITEM changes in docling-ibm-models#171 and docling#3888. Run the Seattle Municipal Code reproduction from the issue, then verify wrapped SECTION_HEADER fragments become one heading with the expected hyphen and word-boundary behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.