docling-project / docling-project/docling

How do I add GPU parallelization for the do_formula_enrichment model?

Open
#1,693 1 comment 1 reaction 0 assignees View on GitHub
question
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

### Question

Right now I am using an elements_batch_size of 2 so it works without have a CUDA out of memory error. But I have 4 GPUs and I would like to use them in parallel and increase the batch size. How would I do that?
...
``` python
from docling_core.types.doc import ImageRefMode, PictureItem, TableItem
from docling.datamodel.settings import settings
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption

pipeline_options = PdfPipelineOptions()
pipeline_options.images_scale = 2.0
pipeline_options.generate_page_images = True
pipeline_options.generate_picture_images = True
pipeline_options.do_formula_enrichment = True
settings.perf.elements_batch_size = 2

doc_converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)
}
)

def get_markdown_from_pdf(input_doc_path, doc_converter=doc_converter):

conv_res = doc_converter.convert(input_doc_path)
translated_markdown = conv_res.document.export_to_markdown(image_mode=ImageRefMode.EMBEDDED)

return translated_markdown
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.