docling-project / docling-project/docling-core
Table serialization loses which cell each value came from
- Dominant language
- HTML
- Stars
- 282
- Forks
- 214
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 21
Description
### Summary
A serialized table is reported as a single `Span` covering the whole `TableItem`, so a value cannot be traced back to the cell it came from. When the same value appears in several rows, provenance resolves to "somewhere in this table".
`TableCell` already carries its own `bbox`, so the information is in the document. It is lost at serialization time, where the layout is delegated to `tabulate` and the returned string no longer records where each cell landed. `Span` has `prov_idx` commented out in `base.py`, so finer granularity looks intended but is not available today.
### Repro
```python
from docling_core.transforms.serializer.markdown import MarkdownDocSerializer
from docling_core.types.doc.document import DoclingDocument, TableCell, TableData
def cell(text, r, c):
return TableCell(
text=text,
start_row_offset_idx=r, end_row_offset_idx=r + 1,
start_col_offset_idx=c, end_col_offset_idx=c + 1,
column_header=(r == 0),
)
rows = [["item", "qty"], ["bolt", "3600"], ["nut", "3600"]]
doc = DoclingDocument(name="repro")
doc.add_table(
data=TableData(
num_rows=3,
num_cols=2,
table_cells=[cell(t, r, c) for r, row in enumerate(rows) for c, t in enumerate(row)],
)
)
res = MarkdownDocSerializer(doc=doc).serialize()
print(res.text)
print(len(res.spans), [type(s.item).__name__ for s in res.spans])
```
```
| item | qty |
|--------|-------|
| bolt | 3600 |
| nut | 3600 |
1 ['TableItem']
```
Both `3600` cells resolve to that single table-wide span, so a citation cannot say which row it came from.
### Proposal
An opt-in `CellSpanTableSerializer(MarkdownTableSerializer)`:
- the text is produced by `super().serialize()` and returned unchanged, so serialized output stays byte-identical;
- the rendered table is then walked to derive the character span of each cell;
- the spans are returned in a `TableSerializationResult` (a `SerializationResult` subclass adding a `cell_spans` field), so nothing changes for existing callers.
One blocker in existing code: `DocSerializer.serialize()` (`common.py:500`) re-wraps every part through `create_ser_result()`, which drops any richer result type an item serializer returned. Returning the part directly when there is exactly one is equivalent in text and spans, and `_PageBreakSerResult` already relies on bypassing that path.
### Environment
- docling-core 2.93.0; behavior confirmed on `main`
- Python 3.11, Windows
I am happy to open a PR with the serializer and tests if this is a direction you would take.
Contributor guide
Research direction
Start with DocSerializer.serialize() in common.py around line 500, the Markdown table serializer, and Span in base.py; verify how create_ser_result() handles the single table result. Done means the serialized text remains byte-identical, existing spans still work, and the table result exposes per-cell spans without being discarded; add focused serializer tests for repeated cell values.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100