docling-project / docling-project/docling-serve

Issue: docling-serve GPU Memory Leak

Open
#72 9 comments 10 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.8k
Forks
340
Avg merge
6d 11h
Merged PRs (30d)
8

Description

Question: docling-serve GPU Memory Leak

Description: docling-serve exhibits steadily increasing GPU memory usage over time when processing a consistent stream of documents, suggesting a memory leak. This leads to potential OOM errors.

Expected: GPU memory should plateau after model loading and remain relatively stable.

Actual: GPU memory usage continuously increases, as seen via nvidia-smi.

Steps to Reproduce:

Set up docling-serve with GPU backend (e.g., pdf_backend: dlparse_v2) and OCR enabled (do_ocr: true).
Send repeated requests to /v1alpha/convert/source or /v1alpha/convert/file with PDF documents. Example curl commands are provided in the original, longer issue description.
Monitor GPU memory usage with nvidia-smi.

How to avoid this for GPU limited resource environments?
I got an h100 with mig of 12GB VRAM but after 4 pdf is starts to throw OOM

docling serve request params
I use docling-serve with this params
params = {
"from_formats": [
"docx",
"pptx",
"html",
"image",
"pdf",
"asciidoc",
"md",
"csv",
"xlsx",
"xml_uspto",
"xml_jats",
"json_docling",
],
"to_formats": ["md"],
"image_export_mode": "placeholder",
"do_ocr": True,
"force_ocr": False,
"ocr_engine": "easyocr",
"ocr_lang": None,
"pdf_backend": "dlparse_v2",
"table_mode": "accurate",
"abort_on_error": False,
"return_as_file": False,
"do_table_structure": True,
"include_images": True,
"images_scale": 2.0,
}

Contributor guide

Open the contributing guide

Research direction

Start by exercising the /v1alpha/convert/source and /v1alpha/convert/file endpoints with repeated PDF requests using GPU, OCR, easyocr, and the listed parameters. Monitor nvidia-smi to determine whether memory keeps growing and whether OOM occurs after several documents. Done means GPU memory plateaus after model loading and repeated processing no longer causes OOM in the reported setup.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.