docling-project / docling-project/docling

DocLayNet classifies structured table regions as picture — content entirely lost in PDF pipeline

Open
#3,409 2 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

Docling version: 2.92.0
Python: 3.12
OS: Windows 11

Problem
When processing technical PDF documents, DocLayNet's layout model frequently classifies structured table regions as picture elements instead of table. Once classified as picture, the region is rendered as in markdown output — the text content is completely lost.

This affects regions that are visually "mixed" layouts, including:

Numbered procedure tables where each row contains: number | text | image/icon
Tables with rowspan/merged cells (indicator state tables, status tables)
Multi-column checklists where each item has an accompanying diagram
Why TableFormerMode.ACCURATE does not help

from docling.datamodel.pipeline_options import TableStructureOptions, TableFormerMode

opts = PdfPipelineOptions()
opts.do_table_structure = True
table_opts = TableStructureOptions()
table_opts.mode = TableFormerMode.ACCURATE
opts.table_structure_options = table_opts
TableFormerMode.ACCURATE only runs on regions that DocLayNet has already classified as table. If DocLayNet classifies a region as picture, TableFormer never runs — ACCURATE mode has no effect.

Minimal reproduction pattern
The problematic PDF layout is a table like this:

| # | Description | Diagram |
|---|-------------------------------------|---------|
| 1 | Check operating environment | [img] |
| 2 | Check for oil leaks | [img] |
| 3 | Check rope/chain condition | [img] |
...
| 14| ... | [img] |
DocLayNet classifies the entire region as picture. Output:

## 5.1.1 Visual Checks

All 14 checklist items are lost.

Current workaround
PictureDescriptionApiOptions with a transcription-focused prompt recovers the content:

from docling.datamodel.pipeline_options import PictureDescriptionApiOptions

opts.do_picture_description = True
opts.picture_description_options = PictureDescriptionApiOptions(
url="http://localhost:8081/v1/chat/completions",
params=dict(model="...", max_tokens=512, temperature=0.0),
prompt="If this is a table or numbered list, transcribe all rows and columns exactly in markdown.",
)
But this requires a running VLM API server and the description quality depends on the VLM — it doesn't produce proper markdown table structure.

Expected / requested behaviour
Better DocLayNet classification: regions that contain predominantly text (even if images are also present per row) should be classified as table or text rather than picture.
Or: a pipeline option to run table structure detection on picture-classified regions as a fallback (e.g., try_table_on_picture=True).
Or: when do_picture_description=True with Granite Vision, use a table-extraction prompt (not just "What is shown in this image?") for regions where the crop aspect ratio/content suggests tabular structure.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.