docling-project / docling-project/docling
Option to keep PICTURE clusters that coincide with a TABLE
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Requested feature
Since 2.103.0 (#3523, extended by #3536 and #3789) a PICTURE cluster whose bbox nearly coincides with
a TABLE cluster is dropped from the `DoclingDocument`. The deduplication is right in the common case
and it saves redundant work, but there is no way to opt out, and dropping the picture also removes
the only signal downstream consumers have that *there is an image there* — including the region's
rendered crop.
Our pipeline uses those crops as a safety net: when TableFormer extracts a table badly, a
vision-model description of the same region recovers the content. Measured on a corporate holiday
calendar: with 2.92 the region survived as a picture and the VLM emitted a clean transcription of the
table; with 2.115 the picture is gone and **two rows of the table disappear from the document**
(TableFormer extracts that table with duplicated columns and two missing rows). The text extraction
itself did not get worse — what we lost is the fallback.
What we would like is a pipeline option, e.g.
`PdfPipelineOptions(drop_pictures_coinciding_with_tables: bool = True)`, keeping today's behaviour as
the default. Keeping the cluster classified as a picture *and* the table would be enough; we do not
need any change in how the table itself is extracted.
### Alternatives
- **Re-render the region ourselves** from the `TableItem` bbox with PyMuPDF and feed that crop to our
vision model. This works and is what we will do if the option is not desirable, but it duplicates
the rendering path Docling already has and it needs the table bbox to be trustworthy, which in this
scenario is precisely what is in doubt.
- **Pin `docling<2.103`.** Not viable: we would give up every later fix, and the pin is already at
2.115.0 with three transitive dependencies pinned alongside it.
- **Post-process `page.predictions.layout.clusters`** before the document is assembled. The dropped
cluster is not available at any public seam we could hook into.
### Docling version
```
docling: 2.115.0
docling-core: 2.87.1
docling-ibm-models: 3.13.2
docling-parse: 7.8.1
```
### Python version
```
Python 3.10.12
```
Contributor guide
Research direction
Start with PdfPipelineOptions and trace the PICTURE/TABLE deduplication introduced in #3523 and extended by #3536 and #3789. Add a regression test showing the option defaults to current dropping behavior and that opting out preserves the picture cluster and its crop alongside the table.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Feature
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 58/100