docling-project / docling-project/docling
TableFormer V2 (TableStructureV2Options) duplicates multi-page reference tables into many overlapping cells; V1 is clean
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
When `TableStructureV2Options` (TableFormer V2) is used to convert a PDF that contains a long, multi-page reference / bibliography list rendered as a table, the resulting `DoclingDocument` emits that table as **dozens of overlapping, sliding-window duplicate rows**. The exact same document converted with `TableStructureOptions` (V1, the historical default) renders the table **once, cleanly**.
On a 24-page research PDF this bloats the Markdown from ~7k words to ~42k words; ~55% of non-blank lines become duplicates, all concentrated in the references table. The duplicate rows are overlapping fragments of the same reference text, e.g.:
```
| ... - Quora, https://www.quora.com/How-do-English-Latin-derived-words-compare-
| ...compare-to-their -Germanic-equivalents-in-terms-of-meaning-and-connotation 37. 3 Ancient ...
| 37. 3 Ancient Public Speaking Secrets for Life-Changing Presentations - Mo...
```
repeated 80+ times with shifting truncation points.
### Reproduction
```python
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import (
PdfPipelineOptions,
TableStructureOptions,
TableStructureV2Options,
)
pdf = "research_paper_with_multipage_bibliography.pdf"
def measure(table_opts_cls):
opts = PdfPipelineOptions()
opts.table_structure_options = table_opts_cls(do_cell_matching=True)
doc = DocumentConverter(
format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=opts)}
).convert(pdf).document
md = doc.export_to_markdown()
lines = [l for l in md.splitlines() if len(l.strip()) > 40]
return len(lines) - len(set(lines)), len(doc.tables)
print("V1:", measure(TableStructureOptions)) # -> (0, 10) clean
print("V2:", measure(TableStructureV2Options)) # -> (452, 10) duplicated
```
### Observed
- **V1** `TableStructureOptions`: **0** duplicate lines, 10 tables.
- **V2** `TableStructureV2Options`: **452** duplicate lines, **same 10 tables**.
- `do_cell_matching=True` and `do_cell_matching=False` both reproduce on V2 (the flag has no effect).
- Identical table *count* in both, so V2 is not finding extra tables — it is duplicating the cell content of the references table.
### Expected
V2 should render the multi-page reference table once, as V1 does.
### Environment
- docling 2.97.0, docling-core 2.78.1, docling-parse 6.2.0, docling-ibm-models 3.13.3
- Python 3.14, macOS 14 (Apple Silicon, MPS)
A minimal repro PDF (a 24-page research paper whose bibliography is rendered as a multi-page table) can be shared on request.
Contributor guide
Assessment
This issue has not been assessed yet.