docling-project / docling-project/docling-core

Table serialization loses which cell each value came from

Open
#738 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
282
Forks
214
Avg merge
2d 15h
Merged PRs (30d)
21

Description

### Summary

A serialized table is reported as a single `Span` covering the whole `TableItem`, so a value cannot be traced back to the cell it came from. When the same value appears in several rows, provenance resolves to "somewhere in this table".

`TableCell` already carries its own `bbox`, so the information is in the document. It is lost at serialization time, where the layout is delegated to `tabulate` and the returned string no longer records where each cell landed. `Span` has `prov_idx` commented out in `base.py`, so finer granularity looks intended but is not available today.

### Repro

```python
from docling_core.transforms.serializer.markdown import MarkdownDocSerializer
from docling_core.types.doc.document import DoclingDocument, TableCell, TableData

def cell(text, r, c):
return TableCell(
text=text,
start_row_offset_idx=r, end_row_offset_idx=r + 1,
start_col_offset_idx=c, end_col_offset_idx=c + 1,
column_header=(r == 0),
)

rows = [["item", "qty"], ["bolt", "3600"], ["nut", "3600"]]
doc = DoclingDocument(name="repro")
doc.add_table(
data=TableData(
num_rows=3,
num_cols=2,
table_cells=[cell(t, r, c) for r, row in enumerate(rows) for c, t in enumerate(row)],
)
)

res = MarkdownDocSerializer(doc=doc).serialize()
print(res.text)
print(len(res.spans), [type(s.item).__name__ for s in res.spans])
```

```
| item | qty |
|--------|-------|
| bolt | 3600 |
| nut | 3600 |
1 ['TableItem']
```

Both `3600` cells resolve to that single table-wide span, so a citation cannot say which row it came from.

### Proposal

An opt-in `CellSpanTableSerializer(MarkdownTableSerializer)`:

- the text is produced by `super().serialize()` and returned unchanged, so serialized output stays byte-identical;
- the rendered table is then walked to derive the character span of each cell;
- the spans are returned in a `TableSerializationResult` (a `SerializationResult` subclass adding a `cell_spans` field), so nothing changes for existing callers.

One blocker in existing code: `DocSerializer.serialize()` (`common.py:500`) re-wraps every part through `create_ser_result()`, which drops any richer result type an item serializer returned. Returning the part directly when there is exactly one is equivalent in text and spans, and `_PageBreakSerResult` already relies on bypassing that path.

### Environment

- docling-core 2.93.0; behavior confirmed on `main`
- Python 3.11, Windows

I am happy to open a PR with the serializer and tests if this is a direction you would take.

Contributor guide

Open the contributing guide

Research direction

Start with DocSerializer.serialize() in common.py around line 500, the Markdown table serializer, and Span in base.py; verify how create_ser_result() handles the single table result. Done means the serialized text remains byte-identical, existing spans still work, and the table result exposes per-cell spans without being discarded; add focused serializer tests for repeated cell values.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.