docling-project / docling-project/docling
When running RapidOCR, getting a lot of warnings and also it's not working in handwritten notes
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
I am getting a lot of unnecessary warnings from ONNXRUNTIME when using Docling with rapidocr backend.
...
### Steps to reproduce
```
from docling.backend.pypdfium2_backend import PyPdfiumDocumentBackend
from docling.datamodel.base_models import InputFormat
from docling.document_converter import (
DocumentConverter as DoclingConverter,
PdfFormatOption,
WordFormatOption,
)
from docling.pipeline.simple_pipeline import SimplePipeline
from docling.pipeline.standard_pdf_pipeline import StandardPdfPipeline
from docling.datamodel.pipeline_options import (
AcceleratorDevice,
AcceleratorOptions,
PdfPipelineOptions,
RapidOcrOptions
)
from src.objects_core import (
ErrorCodes,
file_operation
)
class DoclingDocumentConverter:
"""A class to handle document conversions to markdown format with
optimized image and PDF processing."""
def __init__(
self,
optimize_pdf: bool = True,
pdf_threads: int = 12,
enable_ocr: bool = False,
file_extension: str = "",
):
"""
Initialize the document converter with separate PDF and image optimization options.
Args:
optimize_pdf: Whether to use optimized settings for PDF conversion
pdf_threads: Number of threads for PDF processing
enable_ocr: Whether to enable OCR for PDF files
file_extension: Optional file extension override
"""
self.optimize_pdf = optimize_pdf
self.pdf_threads = pdf_threads
self.enable_ocr = enable_ocr
self.file_extension = file_extension
self._setup_mime_types()
self._initialize_converter()
def _setup_mime_types(self) -> None:
"""Register additional MIME types that might be needed."""
mimetypes.add_type('text/csv', '.csv')
mimetypes.add_type('text/markdown', '.md')
mimetypes.add_type('text/asciidoc', '.asciidoc')
mimetypes.add_type('image/png', '.png')
mimetypes.add_type('image/jpeg', '.jpg')
mimetypes.add_type('image/jpeg', '.jpeg')
def _initialize_converter(self) -> None:
"""Initialize the document converter with appropriate settings."""
pdf_pipeline_options = None
if self.optimize_pdf:
# Configure RapidOCR with GPU acceleration
pdf_pipeline_options = PdfPipelineOptions(ocr_options=RapidOcrOptions())
pdf_pipeline_options.do_ocr = self.enable_ocr
pdf_pipeline_options.do_table_structure = True
pdf_pipeline_options.table_structure_options.do_cell_matching = True
# Configure accelerator to use CUDA explicitly
pdf_pipeline_options.accelerator_options = AcceleratorOptions(
num_threads=self.pdf_threads,
device=AcceleratorDevice.AUTO,
cuda_use_flash_attention2=True # Enable flash attention for faster processing
)
# Create converter instance with separate PDF and image options
self.converter = DoclingConverter(
allowed_formats=[
InputFormat.PDF,
InputFormat.IMAGE,
InputFormat.DOCX,
InputFormat.HTML,
InputFormat.PPTX,
InputFormat.ASCIIDOC,
InputFormat.CSV,
InputFormat.MD,
],
format_options={
InputFormat.PDF: PdfFormatOption(
pipeline_cls=StandardPdfPipeline,
backend=PyPdfiumDocumentBackend,
pipeline_options=pdf_pipeline_options
),
InputFormat.DOCX: WordFormatOption(
pipeline_cls=SimplePipeline
),
},
)
@file_operation(error_code=ErrorCodes.DOCUMENT_CONVERSION_ERROR)
def convert_to_markdown(self, input_path: Union[str, Path]) -> str:
"""
Convert a single file to markdown format.
Args:
input_path: Path to the input file or URL
Returns:
Markdown content as string
Raises:
ValueError: If the file doesn't exist, has an unsupported extension, or conversion fails
"""
# Handle .txt files directly without using docling converter
if self._is_text_file(input_path):
content = self._read_text_file(input_path)
if content is None:
raise ValueError(f"Failed to read text file: {self._safe_log_path(input_path)}")
return content
# For all other file types, use the docling converter
conv_result = self.converter.convert(input_path)
# Get markdown content
markdown_content = conv_result.document.export_to_markdown()
if not markdown_content:
raise ValueError(f"Conversion returned empty content for {self._safe_log_path(input_path)}")
return markdown_content`
```
```
2025-04-03 19:17:43,597 - OrtInferSession - WARNING: DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
WARNING:OrtInferSession:DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
2025-04-03 19:17:43,597 - OrtInferSession - INFO: If you want to use DirectML acceleration, you must do:
INFO:OrtInferSession:If you want to use DirectML acceleration, you must do:
2025-04-03 19:17:43,597 - OrtInferSession - INFO: First, uninstall all onnxruntime pakcages in current environment.
INFO:OrtInferSession:First, uninstall all onnxruntime pakcages in current environment.
2025-04-03 19:17:43,597 - OrtInferSession - INFO: Second, install onnxruntime-directml by `pip install onnxruntime-directml`
INFO:OrtInferSession:Second, install onnxruntime-directml by `pip install onnxruntime-directml`
2025-04-03 19:17:43,598 - OrtInferSession - INFO: Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
INFO:OrtInferSession:Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
2025-04-03 19:17:43,731 - OrtInferSession - WARNING: DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
WARNING:OrtInferSession:DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
2025-04-03 19:17:43,731 - OrtInferSession - INFO: If you want to use DirectML acceleration, you must do:
INFO:OrtInferSession:If you want to use DirectML acceleration, you must do:
2025-04-03 19:17:43,731 - OrtInferSession - INFO: First, uninstall all onnxruntime pakcages in current environment.
INFO:OrtInferSession:First, uninstall all onnxruntime pakcages in current environment.
2025-04-03 19:17:43,731 - OrtInferSession - INFO: Second, install onnxruntime-directml by `pip install onnxruntime-directml`
INFO:OrtInferSession:Second, install onnxruntime-directml by `pip install onnxruntime-directml`
2025-04-03 19:17:43,731 - OrtInferSession - INFO: Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
INFO:OrtInferSession:Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
2025-04-03 19:17:43,765 - OrtInferSession - WARNING: DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
WARNING:OrtInferSession:DmlExecutionProvider is not in available providers (['TensorrtExecutionProvider', 'CUDAExecutionProvider', 'CPUExecutionProvider']). Use TensorrtExecutionProvider inference by default.
2025-04-03 19:17:43,765 - OrtInferSession - INFO: If you want to use DirectML acceleration, you must do:
INFO:OrtInferSession:If you want to use DirectML acceleration, you must do:
2025-04-03 19:17:43,765 - OrtInferSession - INFO: First, uninstall all onnxruntime pakcages in current environment.
INFO:OrtInferSession:First, uninstall all onnxruntime pakcages in current environment.
2025-04-03 19:17:43,765 - OrtInferSession - INFO: Second, install onnxruntime-directml by `pip install onnxruntime-directml`
INFO:OrtInferSession:Second, install onnxruntime-directml by `pip install onnxruntime-directml`
2025-04-03 19:17:43,765 - OrtInferSession - INFO: Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
INFO:OrtInferSession:Third, ensure DmlExecutionProvider is in available providers list. e.g. ['DmlExecutionProvider', 'CPUExecutionProvider']
```
### Docling version
docling 2.28.4
rapidocr-onnxruntime 1.4.4
onnxruntime-gpu 1.21.0
...
### Python version
Python 3.11.9
...
Contributor guide
Assessment
This issue has not been assessed yet.