docling-project / docling-project/docling
Feature request: fallback table detection for picture-classified regions (try_table_on_picture)
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
## Summary
Add an option to run table structure detection on regions classified as `picture` by the
layout model, when those regions exhibit table-like characteristics (cell grid structure,
row/column alignment).
## Current behaviour
All three table structure models (TableFormer V1/V2, Granite Vision) only process regions
labelled `TABLE` or `DOCUMENT_INDEX`. If RT-DETR classifies a region as `PICTURE`, table
structure detection is never invoked — confirmed by Dosu in #[link your issue here].
Related issues: #1653, #767.
## Proposed option
```python
opts = PdfPipelineOptions()
opts.try_table_on_picture = True # attempt TableFormer on picture-classified regions
Motivation / use case
Industrial PDF manuals (e.g. operator manuals for crane systems) contain procedure tables
where each row has: number | text description | diagram/icon. RT-DETR consistently
classifies these as picture because of the embedded images per row.
Result: all checklist text is lost ( in output), making the document
unusable for RAG / information retrieval.
Current workarounds and their limitations
TableFormerMode.ACCURATE — no effect (region never reaches TableFormer)
do_picture_description=True with Granite Vision — produces descriptions, not structured table markdown; improves with v2.93.0 but still not a proper table structure
TableCropsLayoutModel — marks entire page as TABLE, too aggressive for mixed pages
Expected behaviour
When try_table_on_picture=True:
For each picture-classified region, attempt table structure detection
If TableFormer returns a valid table (confidence > threshold), use the table output
Otherwise fall back to picture description /
This is the minimal viable fix. The TableCropsLayoutModel experimental code already
demonstrates the concept is feasible within the existing architecture.
Contributor guide
Assessment
This issue has not been assessed yet.