docling-project / docling-project/docling

TableFormer V1 & V2 mishandling merged rows and columns, irrespective if cell_matching True or False

Open
#3,698 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Bug
The ScheduleTable.pdf test case attached contains a table with both column and row mergeing. We found no combination of TableTransformer V1 or V2, with or without cell matching that would correctly capture the table structure.

Image

The results are summarized here, with the details in the comments

Image

[ScheduleTable.docx](https://github.com/user-attachments/files/29341249/ScheduleTable.docx)

[ScheduleTable.pdf](https://github.com/user-attachments/files/29341447/ScheduleTable.pdf)

The source PDF document contains the same table as a table strucure and as an image of the table.
The PDF was exported from the attached DOCX
...

### Steps to reproduce
Using the ScheduleTable.pdf as the source document
with the following options:

#pipeline_options.table_structure_options = TableStructureV2Options()
pipeline_options.table_structure_options.mode = TableFormerMode.ACCURATE
pipeline_options.table_structure_options.do_cell_matching = True
pipeline_options.images_scale = 2.0
pipeline_options.generate_picture_images = False
pipeline_options.generate_page_images = True

converter = DocumentConverter( format_options={
InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)
})
conversion= converter.convert(source)
doc: DoclingDocument = conversion.document

Four permutations were tested:

V1 vs V2 and cell_matching True vs. False

The 'output' for each of the test runs show the tables as jpeg, html, and otsl and these are available for each of the 4 permutations if that would help (see comment below)

We understand that the table has a challenging structure!!!
...

### Docling version
docling ==2.107.0
...

### Python version

Python 3.12.6
...

Contributor guide

Open the contributing guide

Research direction

Reproduce the issue with ScheduleTable.pdf using DocumentConverter, PdfFormatOption, TableStructureV2Options, and the four TableFormer V1/V2 and cell-matching permutations described. Compare the generated JPEG, HTML, and OTSL outputs with the source table and review the attached results in the comments. Done means the merged rows and columns are represented correctly across the reported configurations.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.