docling-project / docling-project/docling
Language Detection across Backends
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Requested feature
There are various downstream use cases for which it would be desirable to know the language of the given document. For example, when using one of the `PictureDescription` models (e.g., VLM or API), one might want to apply a language-dependent prompt template (instead of using the default `prompt="Describe the image in three sentences. Be consise and accurate."`), such that the language of the picture description aligns with the document language.
Currently, the `DoclingDocument` (and `NodeItem`) object does not support a `language` attribute. One could manually infer the document language before running the document conversion pipeline using [`langdetect`](https://pypi.org/project/langdetect/). This would require the user to extract the document text (or at least the text within the first couple of pages) to have a reliable guess for the document language, which is inefficient (since the text is already extracted during the conversion). In addition, this approach would require separate implementations for every mimetype, which is again inefficient since you already have dedicated backends for different mimetypes.
### Proposed Solution
Integrate the language detection operation into the various backends. Since `langdetect` is quite efficient, this should induce minimal overhead. You could make the language detection optional by adding a pipeline argument like `do_language_detection`. When the language is detected by the respective backend, it could be propagated as part of the `DoclingDocument` or, alternatively, as attribute of each `NoteItem`. For documents that require OCR, you could simply use use specified OCR language or set a default. Potentially, some mimetypes allow to infer the language directly from the file metadata. Alternatively, it might be implemented as an enrichment step via `BaseEnrichmentModel`.
How to elaborate further if required. 🤗
Contributor guide
Assessment
This issue has not been assessed yet.