docling-project / docling-project/docling

OCR'd `TextItem`s inside a TABLE region are never referenced from `body.children` and therefore vanish from `md_content`

Open
#3,473 3 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

## Bug

On scanned form PDFs (thin-lined work-instruction sheets, inspection forms, etc.) the layout model classifies the entire form area as a single `TABLE` cluster. OCR runs and produces correct `TextItem`s for the form-field contents, which are stored in `DoclingDocument.texts[]` with valid `prov.bbox` and `label='text'`.

**None of these `TextItem`s are referenced from `body.children`.** The `MarkdownDocSerializer` iterates the body, so all OCR'd content is missing from `md_content` even though `json_content.texts` contains it.

This is a distinct mechanism from #3461 and #2899: in those issues the items *are* in the body, just in the wrong order. Here the items are not in the body at all.

## Observation against `docling-serve /v1/convert/source/async`

One scanned A4 work-instruction sheet, default options (`do_ocr=true`, `pipeline=standard`, EasyOCR), single representative sample:

| Setting | `texts[]` | `body.children` references to `#/texts/*` | OCR text in `md_content` |
|---|---:|---:|---:|
| `table_cell_matching=true` (default) | 204 | **0** | 0 |
| `table_cell_matching=false` | 204 | **0** | 0 |
| `do_table_structure=false` | 288 | **0** | 0 |
| `pdf_backend=pypdfium2, force_ocr=true, table_cell_matching=false` | 136 | **0** | 0 |

The workarounds suggested in the related dosu-bot answers on #3461 / #2899 / #3194 (`do_cell_matching=False`, `pypdfium2`) do not move the count off zero in this case.

Sorting `json_content.texts[*]` by `(page_no, -bbox.t, bbox.l)` produces a fully-readable rendition of the document — the data is there, it just never reaches the body or the markdown serializer.

## Versions

- `docling-serve` 1.9.0
- `docling` 2.68.0
- `docling-core` 2.x (matching above)
- `docling-jobkit` 1.8.1

Spot-checked against current `main` of `docling-core` (2.51.0): `MarkdownTableSerializer.serialize` still reads exclusively from `item.data.grid` (`markdown.py:371-438`, registered as default at `:707`).

## Where the texts get stranded

- `docling/models/stages/page_assemble/page_assemble_model.py:170-212` — the `TABLE` branch packs the region as a `Table`; text clusters inside the bbox do not go through the `TEXT_ELEM_LABELS` branch and are not emitted into the reading order.
- `docling/models/stages/table_structure/table_structure_model.py:259-275` — TableFormer binds only tokens that match its predicted grid cells. For thin-lined forms the predicted grid does not cover the data rows, so neither `do_matching=True` (token match) nor `do_matching=False` (`page._backend.get_text_in_rect(bbox)` fallback) populates the cells.
- `docling-core/transforms/serializer/markdown.py:371-438` (`MarkdownTableSerializer.serialize`) — reads exclusively from `item.data.grid[r][c].text`; no fallback for `TextItem`s whose bbox lies inside the `TableItem` bbox but that the grid did not absorb.

## Reproduce

Any PDF where the entire content area is classified as a single TABLE cluster by the layout model, with OCR-detectable text inside form fields. Minimal call against `docling-serve`:

```json
{
"sources": [{"kind": "file", "filename": "form.pdf", "base64_string": "..."}],
"options": {
"to_formats": ["md", "json"],
"pipeline": "standard",
"do_ocr": true,
"ocr_lang": ["en"]
}
}
```

Check:

```python
texts = result["document"]["json_content"]["texts"]
body = result["document"]["json_content"]["body"]["children"]
refs = {c["$ref"] for c in body if isinstance(c, dict) and c.get("$ref","").startswith("#/texts/")}
print(len(refs), "of", len(texts), "TextItems reachable from body.children")
# Expected on a thin-lined scanned form: 0 of N
```

## Question

Is this the intended contract — a `TextItem` whose bbox falls inside a `TableItem` is by design only reachable via the table grid? If yes, what is the recommended path to recover that text for Markdown output on scanned forms? If no, is the reading-order pass or the markdown serializer the right place to add a fallback?

## Related

- #3461 — wrong reading order on programmatic table PDF (items in body, wrong order)
- #2899 — fillable PDF content appended at end (items in body, wrong position)
- #3194 — table column order inconsistent (`do_cell_matching=False` workaround)
- #3263 — short isolated paragraphs dropped (outside tables)
- #3153 — multi-line table rows not recognized together

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.