docling-project / docling-project/docling

When running RapidOCR, getting a lot of warnings and also it's not working in handwritten notes

Open
#1,291 1 comment 1 reaction 0 assignees View on GitHub
bug
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Bug
I am getting a lot of unnecessary warnings from ONNXRUNTIME when using Docling with rapidocr backend.
...

### Steps to reproduce
```
from docling.backend.pypdfium2_backend import PyPdfiumDocumentBackend
from docling.datamodel.base_models import InputFormat
from docling.document_converter import (
DocumentConverter as DoclingConverter,
PdfFormatOption,
WordFormatOption,
)
from docling.pipeline.simple_pipeline import SimplePipeline
from docling.pipeline.standard_pdf_pipeline import StandardPdfPipeline
from docling.datamodel.pipeline_options import (
AcceleratorDevice,
AcceleratorOptions,
PdfPipelineOptions,
RapidOcrOptions
)
from src.objects_core import (
ErrorCodes,
file_operation
)
class DoclingDocumentConverter:
"""A class to handle document conversions to markdown format with
optimized image and PDF processing."""

def __init__(
self,
optimize_pdf: bool = True,
pdf_threads: int = 12,
enable_ocr: bool = False,
file_extension: str = "",
):
"""
Initialize the document converter with separate PDF and image optimization options.

Args:
optimize_pdf: Whether to use optimized settings for PDF conversion
pdf_threads: Number of threads for PDF processing
enable_ocr: Whether to enable OCR for PDF files
file_extension: Optional file extension override
"""
self.optimize_pdf = optimize_pdf
self.pdf_threads = pdf_threads
self.enable_ocr = enable_ocr
self.file_extension = file_extension
self._setup_mime_types()
self._initialize_converter()

def _setup_mime_types(self) -> None:
"""Register additional MIME types that might be needed."""
mimetypes.add_type('text/csv', '.csv')
mimetypes.add_type('text/markdown', '.md')
mimetypes.add_type('text/asciidoc', '.asciidoc')
mimetypes.add_type('image/png', '.png')
mimetypes.add_type('image/jpeg', '.jpg')
mimetypes.add_type('image/jpeg', '.jpeg')

def _initialize_converter(self) -> None:
"""Initialize the document converter with appropriate settings."""
pdf_pipeline_options = None
if self.optimize_pdf:
# Configure RapidOCR with GPU acceleration
pdf_pipeline_options = PdfPipelineOptions(ocr_options=RapidOcrOptions())
pdf_pipeline_options.do_ocr = self.enable_ocr
pdf_pipeline_options.do_table_structure = True
pdf_pipeline_options.table_structure_options.do_cell_matching = True

# Configure accelerator to use CUDA explicitly
pdf_pipeline_options.accelerator_options = AcceleratorOptions(
num_threads=self.pdf_threads,
device=AcceleratorDevice.AUTO,
cuda_use_flash_attention2=True # Enable flash attention for faster processing
)

# Create converter instance with separate PDF and image options
self.converter = DoclingConverter(
allowed_formats=[
InputFormat.PDF,
InputFormat.IMAGE,
InputFormat.DOCX,
InputFormat.HTML,
InputFormat.PPTX,
InputFormat.ASCIIDOC,
InputFormat.CSV,
InputFormat.MD,
],
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=StandardPdfPipeline,
backend=PyPdfiumDocumentBackend,
pipeline_options=pdf_pipeline_options
),
InputFormat.DOCX: WordFormatOption(
pipeline_cls=SimplePipeline
),
},
)

@file_operation(error_code=ErrorCodes.DOCUMENT_CONVERSION_ERROR)
def convert_to_markdown(self, input_path: Union[str, Path]) -> str:
"""
Convert a single file to markdown format.

Args:
input_path: Path to the input file or URL

Returns:
Markdown content as string

Raises:
ValueError: If the file doesn't exist, has an unsupported extension, or conversion fails
"""
# Handle .txt files directly without using docling converter
if self._is_text_file(input_path):
content = self._read_text_file(input_path)
if content is None:
raise ValueError(f"Failed to read text file: {self._safe_log_path(input_path)}")
return content

# For all other file types, use the docling converter
conv_result = self.converter.convert(input_path)

# Get markdown content
markdown_content = conv_result.document.export_to_markdown()

if not markdown_content:
raise ValueError(f"Conversion returned empty content for {self._safe_log_path(input_path)}")

return markdown_content`
```

```
2025-04-03 19:17:43,597 - OrtInferSession - WARNING: DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
WARNING:OrtInferSession:DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
2025-04-03 19:17:43,597 - OrtInferSession - INFO: If you want to use DirectML acceleration, you must do:
INFO:OrtInferSession:If you want to use DirectML acceleration, you must do:
2025-04-03 19:17:43,597 - OrtInferSession - INFO: First, uninstall all onnxruntime pakcages in current environment.
INFO:OrtInferSession:First, uninstall all onnxruntime pakcages in current environment.
2025-04-03 19:17:43,597 - OrtInferSession - INFO: Second, install onnxruntime-directml by `pip install onnxruntime-directml`
INFO:OrtInferSession:Second, install onnxruntime-directml by `pip install onnxruntime-directml`
2025-04-03 19:17:43,598 - OrtInferSession - INFO: Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
INFO:OrtInferSession:Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
2025-04-03 19:17:43,731 - OrtInferSession - WARNING: DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
WARNING:OrtInferSession:DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
2025-04-03 19:17:43,731 - OrtInferSession - INFO: If you want to use DirectML acceleration, you must do:
INFO:OrtInferSession:If you want to use DirectML acceleration, you must do:
2025-04-03 19:17:43,731 - OrtInferSession - INFO: First, uninstall all onnxruntime pakcages in current environment.
INFO:OrtInferSession:First, uninstall all onnxruntime pakcages in current environment.
2025-04-03 19:17:43,731 - OrtInferSession - INFO: Second, install onnxruntime-directml by `pip install onnxruntime-directml`
INFO:OrtInferSession:Second, install onnxruntime-directml by `pip install onnxruntime-directml`
2025-04-03 19:17:43,731 - OrtInferSession - INFO: Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
INFO:OrtInferSession:Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
2025-04-03 19:17:43,765 - OrtInferSession - WARNING: DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
WARNING:OrtInferSession:DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
2025-04-03 19:17:43,765 - OrtInferSession - INFO: If you want to use DirectML acceleration, you must do:
INFO:OrtInferSession:If you want to use DirectML acceleration, you must do:
2025-04-03 19:17:43,765 - OrtInferSession - INFO: First, uninstall all onnxruntime pakcages in current environment.
INFO:OrtInferSession:First, uninstall all onnxruntime pakcages in current environment.
2025-04-03 19:17:43,765 - OrtInferSession - INFO: Second, install onnxruntime-directml by `pip install onnxruntime-directml`
INFO:OrtInferSession:Second, install onnxruntime-directml by `pip install onnxruntime-directml`
2025-04-03 19:17:43,765 - OrtInferSession - INFO: Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
INFO:OrtInferSession:Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
```
### Docling version
docling 2.28.4
rapidocr-onnxruntime 1.4.4
onnxruntime-gpu 1.21.0
...

### Python version
Python 3.11.9
...

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.