docling-project / docling-project/docling
Does Docling Support Hindi and/or Sanskrit?
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Question
Does Docling Support Hindi and/or Sanskrit? I did try with EasyOCR and smolvlm_picture_description for image description, but the output text in markdown is gibbrish. Not sure what I am doing wrong
[Book PDF](https://ncert.nic.in/textbook/pdf/jhkr1dd.zip)
Code:
```python
import base64
import re
import textwrap
from io import BytesIO
from pathlib import Path
import os
import numpy as np
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import (
PdfPipelineOptions,
# OCR Models
EasyOcrOptions,
RapidOcrOptions,
TesseractCliOcrOptions,
# VLM Models
smolvlm_picture_description,
granite_picture_description,
)
from docling.document_converter import DocumentConverter, PdfFormatOption
from IPython.display import HTML, display
from ollama import chat
from PIL import Image
pipeline_options = PdfPipelineOptions(
# To create images of each page for visual inspection later.
generate_page_images=True,
# Keep images at their original resolution during processing.
images_scale=1.00,
# To perform Optical Character Recognition (specified in ocr_options),
# ensuring text is extracted even from
# image-based PDFs or scanned sections.
do_ocr=True,
# ocr_options=EasyOcrOptions(lang=["en", "hi"]),
ocr_options=EasyOcrOptions(lang=["hi"]),
# To use a Vision Language Model (VLM)
do_picture_description=True,
picture_description_options=smolvlm_picture_description,
# picture_description_options=granite_picture_description,
# Enrichments
do_formula_enrichment=True, # perform formula OCR, return Latex code
do_table_structure=True,
do_code_enrichment=False,
do_picture_classification=True,
)
converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)
}
)
document_path = Path("./data/hindi/jhkr101.pdf")
result = converter.convert(document_path)
document = result.document
print(document.export_to_text())
```
Output:
Contributor guide
Assessment
This issue has not been assessed yet.