docling-project / docling-project/docling
Layout Model is slow on all CUDA GPUs
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
Regardless of what GPU I use (T4 vs. L4 vs. A100) or num_threads, the layout model processes 1 second per page.
### Steps to reproduce
```python
@extract_images_app.function(image=docling_image, volumes={"/ingestion": vol}, gpu="L4", timeout=3600)
def extract_images(input_pdf, extract_images=True, extract_tables=True):
from docling.datamodel.document import ConversionResult
from docling.document_converter import DocumentConverter, PdfFormatOption
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions, TableFormerMode, AcceleratorDevice, AcceleratorOptions
from docling_core.types.doc import ImageRefMode, PictureItem, TableItem
from PIL import Image
import io
logging.basicConfig(level=logging.INFO)
_log = logging.getLogger(__name__)
artifacts_path = "/root/.cache/docling/models"
pipeline_options = PdfPipelineOptions(artifacts_path=artifacts_path)
pipeline_options.do_ocr = False
pipeline_options.do_table_structure = False
pipeline_options.accelerator_options = AcceleratorOptions(num_threads=256, device=AcceleratorDevice.CUDA)
if extract_images or extract_tables:
pipeline_options.images_scale = 2.0
pipeline_options.generate_page_images = True
pipeline_options.generate_picture_images = True
doc_converter = DocumentConverter(
format_options={InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)}
)
conv_result = doc_converter.convert(input_pdf)
```
Docling version
Version: 2.28.4
Python version
Python 3.10
Contributor guide
Assessment
This issue has not been assessed yet.