docling-project / docling-project/docling

Layout Model is slow on all CUDA GPUs

Open
#1,364 8 comments 1 reaction 0 assignees View on GitHub
bug performance
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Bug

Regardless of what GPU I use (T4 vs. L4 vs. A100) or num_threads, the layout model processes 1 second per page.

### Steps to reproduce
```python
@extract_images_app.function(image=docling_image, volumes={"/ingestion": vol}, gpu="L4", timeout=3600)
def extract_images(input_pdf, extract_images=True, extract_tables=True):
from docling.datamodel.document import ConversionResult
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions, TableFormerMode, AcceleratorDevice, AcceleratorOptions
from docling_core.types.doc import ImageRefMode, PictureItem, TableItem
from PIL import Image
import io

logging.basicConfig(level=logging.INFO)
_log = logging.getLogger(__name__)

artifacts_path = "/root/.cache/docling/models"
pipeline_options = PdfPipelineOptions(artifacts_path=artifacts_path)
pipeline_options.do_ocr = False
pipeline_options.do_table_structure = False
pipeline_options.accelerator_options = AcceleratorOptions(num_threads=256, device=AcceleratorDevice.CUDA)
if extract_images or extract_tables:
pipeline_options.images_scale = 2.0
pipeline_options.generate_page_images = True
pipeline_options.generate_picture_images = True

doc_converter = DocumentConverter(
format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)}
)
conv_result = doc_converter.convert(input_pdf)
```

Docling version
Version: 2.28.4

Python version
Python 3.10

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.