docling-project / docling-project/docling-core

DocLangDocSerializer writes ContentLayer.NOTES/INVISIBLE verbatim, violating the real DocLang 0.7 XSD

Open Beginner friendly
#731 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
282
Forks
214
Avg merge
2d 15h
Merged PRs (30d)
21

Description

## Summary

`DocLangDocSerializer`'s `_create_layer_token` (`docling_core/transforms/serializer/doclang.py`) writes `item.content_layer.value` unconditionally into `` whenever `content_layer != ContentLayer.BODY`. `ContentLayer` has five members (`body`, `furniture`, `background`, `invisible`, `notes`), but the real published DocLang 0.7 XSD (`doclang-project/doclang`, installed `doclang`==0.7.3) only defines three legal enumeration values for that attribute:

```xml











```

Any `DoclingDocument` with content on `ContentLayer.NOTES` or `ContentLayer.INVISIBLE` therefore produces a `.dclg` file that fails real DocLang 0.7 validation (`doclang.validate()`):

```
Element '{https://www.doclang.ai/ns/v0}layer', attribute 'value':
[facet 'enumeration'] The value 'notes' is not an element of the set
{'body', 'background', 'furniture'}.
```

## Reproduction

This is not a contrived case — it fires on ordinary documents:
- docling's own native PPTX backend (`mspowerpoint_backend.py`) extracts real speaker notes as `ContentLayer.NOTES`.
- the OpenDocument backend (`opendocument_backend.py::_get_sheet_content_layer`) marks a hidden spreadsheet sheet `ContentLayer.INVISIBLE`.

Either one, serialized through `DocLangDocSerializer` with default `layers` (all `ContentLayer` members, DocLang's own default per `DocLangParams.layers: set[ContentLayer] = set(ContentLayer)`), produces non-conformant output.

Minimal repro:

```python
from docling_core.types.doc.document import DoclingDocument
from docling_core.types.doc.common.content_layer import ContentLayer
from docling_core.types.doc.labels import DocItemLabel
from docling_core.transforms.serializer.doclang import DocLangDocSerializer, DocLangParams

doc = DoclingDocument(name="probe")
doc.add_text(label=DocItemLabel.TEXT, text="speaker note", content_layer=ContentLayer.NOTES)
xml = DocLangDocSerializer(doc=doc, params=DocLangParams()).serialize().text
print(xml) # contains -- illegal per the real DocLang 0.7 XSD
```

## Root cause

I traced the history: `ContentLayer`'s 5 members were added by #345 (July 2025), for general content-layer filtering (Markdown/HTML export etc.) — unrelated to DocLang. DocLang `` support was added independently 8+ months later by #568 (March 2026), targeting the DocLang v0.5 spec's 3-value vocabulary. The two were never reconciled: #568's own generated documentation only ever demonstrates `body`/`furniture`; `notes`/`invisible` were never mapped onto the spec's legal set. The DocLang backlog tracking issue (#486) marks "content layer (e.g. furniture) not captured in DocLang (#568)" done, but that scope never covered these two values.

## Suggested fix

Clamp at `_create_layer_token`: map `ContentLayer.NOTES` and `ContentLayer.INVISIBLE` onto `furniture` (the spec's own designated fallback for "any supplementary components not contributing to the document's main content" — spec.md's own words, an exact semantic match) rather than writing the raw enum value.

## Environment

- `docling-core` 2.92.0 (also verified against the same `_create_layer_token` shape read directly from the installed package)
- `doclang` (reference toolkit) 0.7.3
- Verified this is not a stale/local XSD copy: the value set has been unchanged from `doclang-project/doclang` v0.5.0 through the current v0.7.3.

Contributor guide

Open the contributing guide

Research direction

Start in docling_core/transforms/serializer/doclang.py at DocLangDocSerializer._create_layer_token, then inspect ContentLayer and the DocLang 0.7 XSD or doclang.validate() behavior described in the issue. Reproduce the NOTES or INVISIBLE case with the provided snippet. Done means serialized output uses only the XSD's legal layer values and validates with doclang.validate().

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
84/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.