docling-project / docling-project/docling-core
Serializers: `meta.language` renders as prose (Markdown/plain text print the pydantic repr, HTML shows a "Meta" block) and `HTMLParams.html_lang` is never emitted
- Dominant language
- HTML
- Stars
- 282
- Forks
- 214
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 23
Description
## Summary
`BaseMeta.language` (`LanguageMetaField`, "the detected human language of the node content, expressed as a BCP 47 code") is the right slot for per-element language, and we now populate it downstream for PDF/UA `/Lang` tagging. But once it is set, every text serializer treats it as content:
1. **Markdown and plain text print the field's pydantic repr.** `MarkdownMetaSerializer._serialize_meta_field` has no `LanguageMetaField` branch, so it falls through to `str(field_val)` (`markdown.py:478` on main) and the document gains a paragraph like `confidence=0.99 created_by='langdetect' code=`. Plain text inherits the same serializer.
2. **HTML renders the language as visible text**, a `` block with `language: de`, instead of a `lang="de"` attribute on the element (WCAG 3.1.2 Language of Parts).
3. **`HTMLParams.html_lang` is declared (`html.py:111`) but never used**: the root element is the hard-coded string `""` (`html.py:1150`), so there is no way to set `` (WCAG 3.1.1 Language of Page) through the API.
Checked against v2.81.0 (installed) and `main` as of 2026-09-03 (v2.94.1); I found no existing issue or PR on these.
## Reproduction
```python
from docling_core.types.doc.base import BoundingBox, CoordOrigin, Size
from docling_core.types.doc.document import BaseMeta, DoclingDocument, LanguageMetaField, ProvenanceItem
from docling_core.types.doc.labels import DocItemLabel, HumanLanguageLabel
from docling_core.transforms.serializer.html import HTMLDocSerializer, HTMLParams
doc = DoclingDocument(name="demo")
doc.add_page(page_no=1, size=Size(width=612, height=792))
item = doc.add_text(
label=DocItemLabel.PARAGRAPH,
text="Dieser Absatz ist auf Deutsch.",
prov=ProvenanceItem(page_no=1, bbox=BoundingBox(l=0, t=100, r=100, b=0, coord_origin=CoordOrigin.BOTTOMLEFT), charspan=(0, 0)),
)
item.meta = BaseMeta(language=LanguageMetaField(code=HumanLanguageLabel.DE, confidence=0.99, created_by="langdetect"))
print(repr(doc.export_to_markdown()))
# "Dieser Absatz ist auf Deutsch.\n\nconfidence=0.99 created_by='langdetect' code="
print(repr(doc.export_to_text()))
# same repr appended as a paragraph
html = HTMLDocSerializer(doc=doc, params=HTMLParams(html_lang="de")).serialize().text
print("lang=" in html) # False — is emitted without the attribute
# body contains: ……
```
## Expected
- Markdown / plain text: a language code is metadata, not content. Either skip `LanguageMetaField` in `_serialize_meta_field` (return `None`) or, at most, emit it only under `mark_meta=True` as `[Language] de`. The current fallthrough to `str()` also means any future `BasePrediction`-derived field leaks its repr the same way, so a safer default for the `else` branch would be to skip unknown field types unless `mark_meta` is set.
`, `
- HTML: emit `lang=""` on the element that carries the text (``, ``, …) when `item.meta.language` differs from the document language, and drop the language row from the meta `` block (or keep it only when the language is the same as the document's, which is never useful).
- HTML root: honour `HTMLParams.html_lang` on ``. Reasonable default when `html_lang` is left at its default: the character-weighted majority of `meta.language` across the body, falling back to `"en"` as today.
## Why it matters
Accessible HTML output needs the page language on the root and the language of parts on the elements; screen readers switch pronunciation on these attributes. Today the only way to get clean text output once `meta.language` is populated is `blocked_meta_names={"language"}`, which callers have to know to pass on every serializer, and `` cannot be set at all without post-processing the string.
## Offer
Happy to open a PR with: the `LanguageMetaField` branch in the Markdown meta serializer (skip unless `mark_meta`), `lang` attributes in the HTML element serializers keyed off `meta.language`, the `` emission from `html_lang` with a majority-language default, and tests for the three cases above. Downstream context: YRA's yra-doc-pipeline populates `meta.language` from a deterministic detector and maps it to PDF/UA `/Lang`; the same data should reach the HTML export.
🤖 Generated with [Claude Code](https://claude.com/claude-code)
Contributor guide
Research direction
Start with MarkdownMetaSerializer._serialize_meta_field at markdown.py:478 and HTMLParams/html.py:111, then inspect the root emission at html.py:1150 and reproduce the three cases in the issue. Done means language metadata no longer becomes text, HTML emits element and root lang attributes as specified, and tests cover Markdown, plain text, and HTML output.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- html, markdown, python
- Domain
- accessibility, documentation
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100