docling-project / docling-project/docling-core
HTML serializer introduces browser-generated markers for list items without a detected marker
- Dominant language
- HTML
- Stars
- 282
- Forks
- 214
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 21
Description
## Description
The HTML serializer can introduce list markers that are not present in the `DoclingDocument`.
This occurs when a `ListGroup` contains a mixture of:
- list items with a detected marker, such as `enumerated=True` and `marker="1."`; and
- list items without a detected marker, such as `enumerated=False` and `marker=""`, where the original numbering remains part of `text`.
A common example is a table of contents where first-level entries are recognized as enumerated list items, but subsection entries such as `1.1` are retained as ordinary text.
The generated list container is selected using only `ListGroup.first_item_is_enumerated(doc)`. If the first item is enumerated, the complete group is emitted as an `
- `.
- ` elements. The browser consequently generates an ordered-list marker for those items, even though no such marker exists in the corresponding `ListItem.marker` field.
This can display additional numbering that was neither detected by Docling nor present in the serialized item text.
## Environment
- `docling-core==2.88.0`
- Python 3.12
- HTML serialization through `HTMLDocSerializer`## Minimal reproduction
```python
from docling_core.transforms.serializer.html import (
HTMLDocSerializer,
HTMLParams,
)
from docling_core.types.doc import DoclingDocumentdoc = DoclingDocument(name="mixed-list")
list_group = doc.add_list_group(name="table-of-contents")
doc.add_list_item(
text="Purpose",
orig="1. Purpose",
enumerated=True,
marker="1.",
parent=list_group,
)doc.add_list_item(
text="1.1 Scope",
orig="1.1 Scope",
enumerated=False,
marker="",
parent=list_group,
)doc.add_list_item(
text="Operation",
orig="2. Operation",
enumerated=True,
marker="2.",
parent=list_group,
)for show_original_marker in (True, False):
result = HTMLDocSerializer(
doc=doc,
params=HTMLParams(
show_original_list_item_marker=show_original_marker,
prettify=False,
),
).serialize()print(
f"show_original_list_item_marker="
f"{show_original_marker}"
)
print(result.text)
```## Actual HTML with `show_original_list_item_marker=True`
```html
- Purpose
- 1.1 Scope
- Operation
```Depending on browser styling, the user sees content equivalent to:
```text
1. Purpose
2. 1.1 Scope
2. Operation
```The `2.` before `1.1 Scope` is generated by the browser because the item is inside an `
- ` and has no explicit `list-style-type`.
- Purpose
- 1.1 Scope
- Operation
- Purpose
- 1.1 Scope
- Operation
## Actual HTML with `show_original_list_item_marker=False`
```html
```This still generates ordered-list markers in the browser:
```text
1. Purpose
2. 1.1 Scope
3. Operation
```Setting `show_original_list_item_marker=False` therefore does not prevent additional browser-generated numbering. It only prevents the serializer from setting the explicit `list-style-type` based on `item.marker`.
## Expected behavior
HTML serialization should be able to preserve the visible content represented by the `DoclingDocument` without adding markers that were not detected.
For the example above, the intended visible output is:
```text
1. Purpose
1.1 Scope
2. Operation
```The rules should be:
1. If `item.marker` contains a detected marker, render that marker.
2. If `item.marker` is empty, do not introduce a browser-generated marker.
3. Preserve `item.text` unchanged.
4. Do not attempt to infer a missing marker from `item.text` during serialization.One possible expected HTML representation is:
```html
```This retains the detected markers while preventing the browser from generating a marker for the item whose `marker` field is empty.
## Relevant implementation behavior
`HTMLListSerializer.serialize()` selects the list container based on the first item:
```python
tag = "ol" if item.first_item_is_enumerated(doc) else "ul"
````ListGroup.first_item_is_enumerated()` checks only the first child:
```python
def first_item_is_enumerated(self, doc: "DoclingDocument"):
return (
len(self.children) > 0
and isinstance(
first_child := self.children[0].resolve(doc),
ListItem,
)
and first_child.enumerated
)
````HTMLTextSerializer.serialize()` currently applies an explicit marker style only when both of the following are true:
- `show_original_list_item_marker` is enabled; and
- `item.marker` is non-empty.The relevant logic is:
```python
attrs=(
{
"style": (
f"list-style-type: '{item.marker} ';"
)
}
if params.show_original_list_item_marker
and item.marker
else {}
)
```When `item.marker` is empty, the resulting `attrs={}` permits the browser to use the default marker inherited from `
- ` or `
- `.
## Potential solution 1: suppress the marker per item
When original-marker rendering is enabled, explicitly use `list-style-type: none` for items without a detected marker:
```python
if params.show_original_list_item_marker:
attrs = {
"style": (
f"list-style-type: '{item.marker} ';"
if item.marker
else "list-style-type: none;"
)
}
else:
attrs = {}
```The existing tag creation could then use:
```python
text = get_html_tag_with_text_direction(
html_tag="li",
text=text,
attrs=attrs,
)
```This is a small and local change. It prevents the parent list container from introducing an implicit marker when the `DoclingDocument` has no marker for that item.
## Potential solution 2: add a separate parameter
Changing the existing behavior of `show_original_list_item_marker=True` may affect users who currently rely on browser-generated fallback markers.
For backward compatibility, a separate parameter could be introduced:
```python
suppress_default_list_item_marker: bool = False
```Possible behavior:
```python
if params.show_original_list_item_marker and item.marker:
attrs = {
"style": (
f"list-style-type: '{item.marker} ';"
)
}
elif params.suppress_default_list_item_marker:
attrs = {"style": "list-style-type: none;"}
else:
attrs = {}
```This would allow fidelity-oriented consumers to request:
```python
HTMLParams(
show_original_list_item_marker=True,
suppress_default_list_item_marker=True,
)
```The resulting contract would be:
- show a marker when one is present in `ListItem.marker`;
- show no marker when `ListItem.marker` is empty;
- never derive or introduce numbering that is not represented in the document item.## Potential solution 3: serializer-level marker strategy
A more extensible alternative would be a marker policy enum, for example:
```python
class HTMLListMarkerMode(str, Enum):
AUTO = "auto"
ORIGINAL = "original"
NONE = "none"
```Possible meanings:
- `AUTO`: retain the current HTML/browser behavior.
- `ORIGINAL`: show `item.marker` when present and suppress the marker otherwise.
- `NONE`: suppress all list markers.This would make the intended behavior clearer than combining multiple booleans.
List items without an explicit marker are then emitted as plain `
Contributor guide
Research direction
Start with the minimal reproduction and inspect HTMLListSerializer.serialize and HTMLTextSerializer.serialize, including the HTMLParams marker settings. Done means detected markers remain visible, empty ListItem.marker values produce no browser-generated marker, and item.text remains unchanged; validate both show_original_list_item_marker settings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 25/100