docling-project / docling-project/docling

Does Docling Support Hindi and/or Sanskrit?

Open
#2,336 2 comments 0 reactions 0 assignees View on GitHub
question triage/close-stale
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Question
Does Docling Support Hindi and/or Sanskrit? I did try with EasyOCR and smolvlm_picture_description for image description, but the output text in markdown is gibbrish. Not sure what I am doing wrong

[Book PDF](https://ncert.nic.in/textbook/pdf/jhkr1dd.zip)

Code:

```python
import base64
import re
import textwrap
from io import BytesIO
from pathlib import Path
import os

import numpy as np
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import (
PdfPipelineOptions,

# OCR Models
EasyOcrOptions,
RapidOcrOptions,
TesseractCliOcrOptions,

# VLM Models
smolvlm_picture_description,
granite_picture_description,
)
from docling.document_converter import DocumentConverter, PdfFormatOption
from IPython.display import HTML, display
from ollama import chat
from PIL import Image

pipeline_options = PdfPipelineOptions(
# To create images of each page for visual inspection later.
generate_page_images=True,

# Keep images at their original resolution during processing.
images_scale=1.00,

# To perform Optical Character Recognition (specified in ocr_options),
# ensuring text is extracted even from
# image-based PDFs or scanned sections.
do_ocr=True,
# ocr_options=EasyOcrOptions(lang=["en", "hi"]),
ocr_options=EasyOcrOptions(lang=["hi"]),

# To use a Vision Language Model (VLM)
do_picture_description=True,
picture_description_options=smolvlm_picture_description,
# picture_description_options=granite_picture_description,

# Enrichments
do_formula_enrichment=True, # perform formula OCR, return Latex code
do_table_structure=True,
do_code_enrichment=False,
do_picture_classification=True,
)

converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)
}
)

document_path = Path("./data/hindi/jhkr101.pdf")
result = converter.convert(document_path)
document = result.document

print(document.export_to_text())
```

Output:

Image

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.