docling-project / docling-project/docling
DocLayNet classifies structured table regions as picture — content entirely lost in PDF pipeline
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
Docling version: 2.92.0
Python: 3.12
OS: Windows 11
Problem
When processing technical PDF documents, DocLayNet's layout model frequently classifies structured table regions as picture elements instead of table. Once classified as picture, the region is rendered as in markdown output — the text content is completely lost.
This affects regions that are visually "mixed" layouts, including:
Numbered procedure tables where each row contains: number | text | image/icon
Tables with rowspan/merged cells (indicator state tables, status tables)
Multi-column checklists where each item has an accompanying diagram
Why TableFormerMode.ACCURATE does not help
from docling.datamodel.pipeline_options import TableStructureOptions, TableFormerMode
opts = PdfPipelineOptions()
opts.do_table_structure = True
table_opts = TableStructureOptions()
table_opts.mode = TableFormerMode.ACCURATE
opts.table_structure_options = table_opts
TableFormerMode.ACCURATE only runs on regions that DocLayNet has already classified as table. If DocLayNet classifies a region as picture, TableFormer never runs — ACCURATE mode has no effect.
Minimal reproduction pattern
The problematic PDF layout is a table like this:
| # | Description | Diagram |
|---|-------------------------------------|---------|
| 1 | Check operating environment | [img] |
| 2 | Check for oil leaks | [img] |
| 3 | Check rope/chain condition | [img] |
...
| 14| ... | [img] |
DocLayNet classifies the entire region as picture. Output:
## 5.1.1 Visual Checks
All 14 checklist items are lost.
Current workaround
PictureDescriptionApiOptions with a transcription-focused prompt recovers the content:
from docling.datamodel.pipeline_options import PictureDescriptionApiOptions
opts.do_picture_description = True
opts.picture_description_options = PictureDescriptionApiOptions(
url="http://localhost:8081/v1/chat/completions",
params=dict(model="...", max_tokens=512, temperature=0.0),
prompt="If this is a table or numbered list, transcribe all rows and columns exactly in markdown.",
)
But this requires a running VLM API server and the description quality depends on the VLM — it doesn't produce proper markdown table structure.
Expected / requested behaviour
Better DocLayNet classification: regions that contain predominantly text (even if images are also present per row) should be classified as table or text rather than picture.
Or: a pipeline option to run table structure detection on picture-classified regions as a fallback (e.g., try_table_on_picture=True).
Or: when do_picture_description=True with Granite Vision, use a table-extraction prompt (not just "What is shown in this image?") for regions where the crop aspect ratio/content suggests tabular structure.
Contributor guide
Assessment
This issue has not been assessed yet.