docling-project / docling-project/docling
Prefer native structured math (MathML/OMML) over formula reconstruction
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
## Feature request: extract embedded MathML / native structured equations before formula reconstruction
### Description
Does Docling currently detect and extract a structured mathematical representation that is already embedded in the source document, instead of reconstructing the formula visually using the formula enrichment model?
In particular, modern accessible/tagged PDFs can contain a MathML representation associated with a `Formula` structure element. Depending on the producer, this may be represented through mechanisms such as:
- an associated file / embedded XML stream containing MathML;
- MathML-related structure elements in the PDF structure tree;
- producer-specific attributes containing MathML (for example, Microsoft documents PDF/UA output containing an `MSFT_MathML` attribute).
If this information is available in the PDF, it seems preferable to extract it directly rather than:
1. detecting the rendered formula;
2. interpreting the visual glyphs;
3. reconstructing a LaTeX representation with the formula enrichment model.
The embedded representation should generally be more faithful to the author's original mathematical structure.
### Current behavior
The Docling documentation describes formula understanding as analyzing equation formulas and extracting their LaTeX representation.
As far as I can tell, it is not clear whether Docling first checks the PDF structure tree / associated files for an existing MathML representation.
Could you clarify whether this is currently supported?
### Proposed behavior
When parsing a PDF, Docling could use the following priority:
1. **Native embedded MathML**, when explicitly associated with the formula in the PDF.
2. Other structured mathematical metadata exposed by the PDF producer.
3. Existing formula understanding / visual reconstruction as a fallback.
For example, conceptually:
```text
PDF Formula structure element
|
+-- MathML available?
| |
| +-- yes -> parse and preserve MathML
|
+-- no -> existing formula enrichment
|
+-- reconstructed LaTeX
```
Ideally the resulting `FormulaItem` could retain both representations when available, e.g. something conceptually similar to:
```python
FormulaItem(
text="...",
latex=r"\frac{x}{y}",
mathml="",
)
```
The precise data model is of course up to Docling.
### Why this would be useful
This would avoid unnecessary lossy formula recognition for born-digital accessible documents.
Potential advantages:
- higher fidelity;
- no dependence on formula-recognition model accuracy;
- better handling of complex notation;
- preservation of accessibility information;
- easier conversion to HTML;
- better downstream use in scientific-document pipelines;
- possibility to preserve both MathML and LaTeX representations.
It could also substantially improve extraction from documents produced specifically with accessible-math workflows such as modern tagged LaTeX/PDF.
### DOCX
There appears to be an analogous opportunity for DOCX, although the underlying representation is different.
Word equations in `.docx` are normally stored as **OMML (Office Math Markup Language)** rather than MathML.
Docling already appears to parse OMML and convert it to LaTeX. Would it make sense for Docling to also preserve the native structured representation, and/or expose a MathML conversion alongside the generated LaTeX?
Conceptually the desired behavior could therefore be:
```text
PDF
embedded MathML -> FormulaItem.mathml
-> optionally derive LaTeX
DOCX
native OMML -> preserve structured math
-> MathML
-> LaTeX
```
rather than converting every source immediately into only LaTeX.
Microsoft documents an OMML → MathML mapping, so MathML could potentially provide a common normalized representation across PDF and DOCX inputs.
### Suggested questions
1. Does the PDF backend currently inspect tagged-PDF `Formula` elements for associated MathML?
2. Does it inspect PDF associated files / embedded XML associated with formulas?
3. Does it recognize producer-specific MathML attributes such as `MSFT_MathML`?
4. If MathML is present, can it be exposed in `FormulaItem` rather than running formula enrichment?
5. For DOCX, could the existing OMML parser expose MathML in addition to LaTeX?
6. Would Docling be interested in using MathML as a common structured representation for native equations across document formats?
### References
- LaTeX Tagged PDF project: accessible mathematics can associate MathML with `Formula` structure elements.
- PDF Association / Well-Tagged PDF guidance: MathML may be represented using associated files and/or MathML structure elements.
- Microsoft 365 MathML documentation: Office Math/OMML can be exported to MathML, and PDF/UA export can expose MathML for equations.
- Docling formula enrichment documentation: currently describes extraction of a LaTeX representation.
Contributor guide
Assessment
This issue has not been assessed yet.