docling-project / docling-project/docling

Option to keep PICTURE clusters that coincide with a TABLE

Open
#3,922 0 comments 1 reaction 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Requested feature

Since 2.103.0 (#3523, extended by #3536 and #3789) a PICTURE cluster whose bbox nearly coincides with
a TABLE cluster is dropped from the `DoclingDocument`. The deduplication is right in the common case
and it saves redundant work, but there is no way to opt out, and dropping the picture also removes
the only signal downstream consumers have that *there is an image there* — including the region's
rendered crop.

Our pipeline uses those crops as a safety net: when TableFormer extracts a table badly, a
vision-model description of the same region recovers the content. Measured on a corporate holiday
calendar: with 2.92 the region survived as a picture and the VLM emitted a clean transcription of the
table; with 2.115 the picture is gone and **two rows of the table disappear from the document**
(TableFormer extracts that table with duplicated columns and two missing rows). The text extraction
itself did not get worse — what we lost is the fallback.

What we would like is a pipeline option, e.g.
`PdfPipelineOptions(drop_pictures_coinciding_with_tables: bool = True)`, keeping today's behaviour as
the default. Keeping the cluster classified as a picture *and* the table would be enough; we do not
need any change in how the table itself is extracted.

### Alternatives

- **Re-render the region ourselves** from the `TableItem` bbox with PyMuPDF and feed that crop to our
vision model. This works and is what we will do if the option is not desirable, but it duplicates
the rendering path Docling already has and it needs the table bbox to be trustworthy, which in this
scenario is precisely what is in doubt.
- **Pin `docling<2.103`.** Not viable: we would give up every later fix, and the pin is already at
2.115.0 with three transitive dependencies pinned alongside it.
- **Post-process `page.predictions.layout.clusters`** before the document is assembled. The dropped
cluster is not available at any public seam we could hook into.

### Docling version

```
docling: 2.115.0
docling-core: 2.87.1
docling-ibm-models: 3.13.2
docling-parse: 7.8.1
```

### Python version

```
Python 3.10.12
```

Contributor guide

Open the contributing guide

Research direction

Start with PdfPipelineOptions and trace the PICTURE/TABLE deduplication introduced in #3523 and extended by #3536 and #3789. Add a regression test showing the option defaults to current dropping behavior and that opting out preserves the picture cluster and its crop alongside the table.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.