docling-project / docling-project/docling
How do I add GPU parallelization for the do_formula_enrichment model?
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Question
Right now I am using an elements_batch_size of 2 so it works without have a CUDA out of memory error. But I have 4 GPUs and I would like to use them in parallel and increase the batch size. How would I do that?
...
``` python
from docling_core.types.doc import ImageRefMode, PictureItem, TableItem
from docling.datamodel.settings import settings
from docling.datamodel.base_models import InputFormat
from docling.datamodel.pipeline_options import PdfPipelineOptions
from docling.document_converter import DocumentConverter, PdfFormatOption
pipeline_options = PdfPipelineOptions()
pipeline_options.images_scale = 2.0
pipeline_options.generate_page_images = True
pipeline_options.generate_picture_images = True
pipeline_options.do_formula_enrichment = True
settings.perf.elements_batch_size = 2
doc_converter = DocumentConverter(
format_options={
InputFormat.PDF: PdfFormatOption(pipeline_options=pipeline_options)
}
)
def get_markdown_from_pdf(input_doc_path, doc_converter=doc_converter):
conv_res = doc_converter.convert(input_doc_path)
translated_markdown = conv_res.document.export_to_markdown(image_mode=ImageRefMode.EMBEDDED)
return translated_markdown
```
Contributor guide
Assessment
This issue has not been assessed yet.