docling-project / docling-project/docling-core

HTML serializer introduces browser-generated markers for list items without a detected marker

Open
#699 0 comments 0 reactions 0 assignees Claimed by @julianfesel View on GitHub
Dominant language
HTML
Stars
282
Forks
214
Avg merge
2d 15h
Merged PRs (30d)
21

Description

## Description

The HTML serializer can introduce list markers that are not present in the `DoclingDocument`.

This occurs when a `ListGroup` contains a mixture of:

- list items with a detected marker, such as `enumerated=True` and `marker="1."`; and
- list items without a detected marker, such as `enumerated=False` and `marker=""`, where the original numbering remains part of `text`.

A common example is a table of contents where first-level entries are recognized as enumerated list items, but subsection entries such as `1.1` are retained as ordinary text.

The generated list container is selected using only `ListGroup.first_item_is_enumerated(doc)`. If the first item is enumerated, the complete group is emitted as an `

    `.

    List items without an explicit marker are then emitted as plain `

  1. ` elements. The browser consequently generates an ordered-list marker for those items, even though no such marker exists in the corresponding `ListItem.marker` field.

    This can display additional numbering that was neither detected by Docling nor present in the serialized item text.

    ## Environment

    - `docling-core==2.88.0`
    - Python 3.12
    - HTML serialization through `HTMLDocSerializer`

    ## Minimal reproduction

    ```python
    from docling_core.transforms.serializer.html import (
    HTMLDocSerializer,
    HTMLParams,
    )
    from docling_core.types.doc import DoclingDocument

    doc = DoclingDocument(name="mixed-list")

    list_group = doc.add_list_group(name="table-of-contents")

    doc.add_list_item(
    text="Purpose",
    orig="1. Purpose",
    enumerated=True,
    marker="1.",
    parent=list_group,
    )

    doc.add_list_item(
    text="1.1 Scope",
    orig="1.1 Scope",
    enumerated=False,
    marker="",
    parent=list_group,
    )

    doc.add_list_item(
    text="Operation",
    orig="2. Operation",
    enumerated=True,
    marker="2.",
    parent=list_group,
    )

    for show_original_marker in (True, False):
    result = HTMLDocSerializer(
    doc=doc,
    params=HTMLParams(
    show_original_list_item_marker=show_original_marker,
    prettify=False,
    ),
    ).serialize()

    print(
    f"show_original_list_item_marker="
    f"{show_original_marker}"
    )
    print(result.text)
    ```

    ## Actual HTML with `show_original_list_item_marker=True`

    ```html


    1. Purpose

    2. 1.1 Scope

    3. Operation


    ```

    Depending on browser styling, the user sees content equivalent to:

    ```text
    1. Purpose
    2. 1.1 Scope
    2. Operation
    ```

    The `2.` before `1.1 Scope` is generated by the browser because the item is inside an `

      ` and has no explicit `list-style-type`.

      ## Actual HTML with `show_original_list_item_marker=False`

      ```html


      1. Purpose

      2. 1.1 Scope

      3. Operation


      ```

      This still generates ordered-list markers in the browser:

      ```text
      1. Purpose
      2. 1.1 Scope
      3. Operation
      ```

      Setting `show_original_list_item_marker=False` therefore does not prevent additional browser-generated numbering. It only prevents the serializer from setting the explicit `list-style-type` based on `item.marker`.

      ## Expected behavior

      HTML serialization should be able to preserve the visible content represented by the `DoclingDocument` without adding markers that were not detected.

      For the example above, the intended visible output is:

      ```text
      1. Purpose
      1.1 Scope
      2. Operation
      ```

      The rules should be:

      1. If `item.marker` contains a detected marker, render that marker.
      2. If `item.marker` is empty, do not introduce a browser-generated marker.
      3. Preserve `item.text` unchanged.
      4. Do not attempt to infer a missing marker from `item.text` during serialization.

      One possible expected HTML representation is:

      ```html


      1. Purpose

      2. 1.1 Scope

      3. Operation


      ```

      This retains the detected markers while preventing the browser from generating a marker for the item whose `marker` field is empty.

      ## Relevant implementation behavior

      `HTMLListSerializer.serialize()` selects the list container based on the first item:

      ```python
      tag = "ol" if item.first_item_is_enumerated(doc) else "ul"
      ```

      `ListGroup.first_item_is_enumerated()` checks only the first child:

      ```python
      def first_item_is_enumerated(self, doc: "DoclingDocument"):
      return (
      len(self.children) > 0
      and isinstance(
      first_child := self.children[0].resolve(doc),
      ListItem,
      )
      and first_child.enumerated
      )
      ```

      `HTMLTextSerializer.serialize()` currently applies an explicit marker style only when both of the following are true:

      - `show_original_list_item_marker` is enabled; and
      - `item.marker` is non-empty.

      The relevant logic is:

      ```python
      attrs=(
      {
      "style": (
      f"list-style-type: '{item.marker} ';"
      )
      }
      if params.show_original_list_item_marker
      and item.marker
      else {}
      )
      ```

      When `item.marker` is empty, the resulting `attrs={}` permits the browser to use the default marker inherited from `

        ` or `
          `.

          ## Potential solution 1: suppress the marker per item

          When original-marker rendering is enabled, explicitly use `list-style-type: none` for items without a detected marker:

          ```python
          if params.show_original_list_item_marker:
          attrs = {
          "style": (
          f"list-style-type: '{item.marker} ';"
          if item.marker
          else "list-style-type: none;"
          )
          }
          else:
          attrs = {}
          ```

          The existing tag creation could then use:

          ```python
          text = get_html_tag_with_text_direction(
          html_tag="li",
          text=text,
          attrs=attrs,
          )
          ```

          This is a small and local change. It prevents the parent list container from introducing an implicit marker when the `DoclingDocument` has no marker for that item.

          ## Potential solution 2: add a separate parameter

          Changing the existing behavior of `show_original_list_item_marker=True` may affect users who currently rely on browser-generated fallback markers.

          For backward compatibility, a separate parameter could be introduced:

          ```python
          suppress_default_list_item_marker: bool = False
          ```

          Possible behavior:

          ```python
          if params.show_original_list_item_marker and item.marker:
          attrs = {
          "style": (
          f"list-style-type: '{item.marker} ';"
          )
          }
          elif params.suppress_default_list_item_marker:
          attrs = {"style": "list-style-type: none;"}
          else:
          attrs = {}
          ```

          This would allow fidelity-oriented consumers to request:

          ```python
          HTMLParams(
          show_original_list_item_marker=True,
          suppress_default_list_item_marker=True,
          )
          ```

          The resulting contract would be:

          - show a marker when one is present in `ListItem.marker`;
          - show no marker when `ListItem.marker` is empty;
          - never derive or introduce numbering that is not represented in the document item.

          ## Potential solution 3: serializer-level marker strategy

          A more extensible alternative would be a marker policy enum, for example:

          ```python
          class HTMLListMarkerMode(str, Enum):
          AUTO = "auto"
          ORIGINAL = "original"
          NONE = "none"
          ```

          Possible meanings:

          - `AUTO`: retain the current HTML/browser behavior.
          - `ORIGINAL`: show `item.marker` when present and suppress the marker otherwise.
          - `NONE`: suppress all list markers.

          This would make the intended behavior clearer than combining multiple booleans.

Contributor guide

Open the contributing guide

Research direction

Start with the minimal reproduction and inspect HTMLListSerializer.serialize and HTMLTextSerializer.serialize, including the HTMLParams marker settings. Done means detected markers remain visible, empty ListItem.marker values produce no browser-generated marker, and item.text remains unchanged; validate both show_original_list_item_marker settings.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.