docling-project / docling-project/docling

Language Detection across Backends

Open
#1,370 2 comments 2 reactions 2 assignees Claimed by @dolfim-ibm View on GitHub
enhancement language support triage/close-stale
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Requested feature
There are various downstream use cases for which it would be desirable to know the language of the given document. For example, when using one of the `PictureDescription` models (e.g., VLM or API), one might want to apply a language-dependent prompt template (instead of using the default `prompt="Describe the image in three sentences. Be consise and accurate."`), such that the language of the picture description aligns with the document language.

Currently, the `DoclingDocument` (and `NodeItem`) object does not support a `language` attribute. One could manually infer the document language before running the document conversion pipeline using [`langdetect`](https://pypi.org/project/langdetect/). This would require the user to extract the document text (or at least the text within the first couple of pages) to have a reliable guess for the document language, which is inefficient (since the text is already extracted during the conversion). In addition, this approach would require separate implementations for every mimetype, which is again inefficient since you already have dedicated backends for different mimetypes.

### Proposed Solution
Integrate the language detection operation into the various backends. Since `langdetect` is quite efficient, this should induce minimal overhead. You could make the language detection optional by adding a pipeline argument like `do_language_detection`. When the language is detected by the respective backend, it could be propagated as part of the `DoclingDocument` or, alternatively, as attribute of each `NoteItem`. For documents that require OCR, you could simply use use specified OCR language or set a default. Potentially, some mimetypes allow to infer the language directly from the file metadata. Alternatively, it might be implemented as an enrichment step via `BaseEnrichmentModel`.

How to elaborate further if required. 🤗

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.