docling-project / docling-project/docling

Words assigned to a table by the layout stage vanish from the output when TableFormer's grid does not cover them

Open
#4,181 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Bug

When the table structure model returns a grid that leaves a PDF word outside every row band and every column band, `MatchingPostProcessor._pick_orphan_cells` in docling-ibm-models drops that word, and because the layout stage already assigned it to the table cluster, docling never emits it as text either. The word is absent from the `DoclingDocument`, the Markdown and the JSON export. On a dense public utility matrix this loses 604 of 615 words on one page, and whole pages of content across the document, with only a log warning.

Mechanism, from reading the code and running the reproduction below:

1. The layout model returns one `table` cluster that also covers the form header above the matrix (`Project Owner`, `Project No.`, `Project Description`, `Highway or Route`). It also predicts a `key_value_region` for that header at confidence 0.33, which the layout post-processor discards (threshold 0.45), so the header lines are folded into the table cluster.
2. On that crop TableFormer's greedy decoder emits `ched lcel lcel lcel ched lcel lcel lcel ...` and keeps emitting `lcel` until `max_steps` (1024) with no `nl` token. The prediction is accepted as a valid one-row structure. Cropping the same cluster below the form header lets the unchanged model decode a real structure and place every word, so the collapse is input-sensitive rather than fixed per page.
3. `_pick_orphan_cells` snaps orphans that match one band to the nearest row or column (the fallbacks added for docling-ibm-models #28 and #3402) and drops the rest with the warning `N of M pdf cells matched neither a row nor a column band ... and were dropped from the table`.
4. `PageAssembleModel` emits text only for clusters with text labels; words in the table cluster that no table cell references are never emitted.

Steps 3 and 4 together turn a structure error into silent content loss. The same loss happens at a smaller scale whenever the predicted grid is narrower than the text region: on the pages of this document that decode properly, 7 to 19 words each still disappear (right-aligned values, rotated column headers).

Expected: every word the parser found on the page is present somewhere in the document, in a table cell or as a text item, even when the table structure is imperfect.

Related: #3402 (last part of a table missing), #3473 (OCR text inside a table region vanishing), docling-ibm-models #28. The layout box handed to TableFormer here also includes shapes that docling-parse reports in raw PDF coordinates (docling-parse #343), which matters if anyone tries to cut the cluster to its drawn frame.

Two places could close the gap; the second needs no model change:

- docling-ibm-models: attach an orphan that matches neither band to the nearest table cell by centroid distance, as the existing one-axis fallbacks do, or return the unmatched pdf cell ids in `predict_details`.
- docling: after the table stage, emit every word of a table cluster that no table cell references as a text item (or attach it to the nearest cell), so the document never loses content the parser found.

A decoder guard would also help diagnosis: a sequence that reaches `max_steps` with zero `nl` tokens cannot be a valid table, and today it is passed downstream as a one-row structure. Noticed while reading the code: `tm_config.json` sets `beam_size: 5`, but `predict()` decodes with greedy argmax and never uses it.

### Steps to reproduce

Public document: WSDOT contract 9424, Appendix U2, Existing Utility Listing, (SHA-256 `e619a4abf6044ee17415d4f6933ec9ef674c67735befb172a8d6da5a691e4b9a`), page 6, a 38-row by 28-column matrix with about 615 words. OCR is off because the page has native text; default TableFormer (accurate, cell matching) and the default layout model.

```python
import logging
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption

logging.basicConfig(level=logging.WARNING)
options = PdfPipelineOptions(do_ocr=False)
converter = DocumentConverter(
format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=options)}
)
doc = converter.convert("U2-Existing Utility Listing.pdf", page_range=(6, 6)).document
print([(t.data.num_rows, t.data.num_cols, len(t.data.table_cells)) for t in doc.tables])
print(len(doc.texts))
print(sum(len(c.text.split()) for t in doc.tables for c in t.data.table_cells))
```

Observed:

```
MatchingPostProcessor WARNING 604 of 615 pdf cells matched neither a row nor a column band of the 2x5 grid and were dropped from the table
[(1, 1, 1)]
0
11
```

The output is one table with one cell, `Project Owner: WSDOT Project No. : Project Description: SR509 Completion Project`, no text items and 11 words. Pages 2, 3, 4, 7, 8 and 10 of the same file behave the same way (604 to 1016 words dropped each); pages 1, 5, 9 and 11 lose 7 to 19 words each.

Running docling's own TableFormer on the saved page-6 cluster reproduces the collapse deterministically on CPU in about 15 seconds; the decoded sequence is 1023 tokens long with no row break.

### Docling version

Docling version: 2.126.0
Docling Core version: 2.95.0
Docling IBM Models version: 4.0.2
docling-parse 7.17.0

### Python version

Python 3.12.12 (macOS 26.6, Apple M1 Pro, CPU inference)

Contributor guide

Open the contributing guide

Research direction

Start by running the page-6 reproduction with the documented Docling versions, then inspect MatchingPostProcessor._pick_orphan_cells and PageAssembleModel's handling of table-cluster words. Compare the dropped pdf cells with the resulting DoclingDocument, Markdown, and JSON. Done means every parsed word is represented in a table cell or as a text item, with regression coverage for imperfect table structures.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
ai, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.