docling-project / docling-project/docling-serve
Issue: docling-serve GPU Memory Leak
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 340
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 8
Description
Question: docling-serve GPU Memory Leak
Description: docling-serve exhibits steadily increasing GPU memory usage over time when processing a consistent stream of documents, suggesting a memory leak. This leads to potential OOM errors.
Expected: GPU memory should plateau after model loading and remain relatively stable.
Actual: GPU memory usage continuously increases, as seen via nvidia-smi.
Steps to Reproduce:
Set up docling-serve with GPU backend (e.g., pdf_backend: dlparse_v2) and OCR enabled (do_ocr: true).
Send repeated requests to /v1alpha/convert/source or /v1alpha/convert/file with PDF documents. Example curl commands are provided in the original, longer issue description.
Monitor GPU memory usage with nvidia-smi.
How to avoid this for GPU limited resource environments?
I got an h100 with mig of 12GB VRAM but after 4 pdf is starts to throw OOM
docling serve request params
I use docling-serve with this params
params = {
"from_formats": [
"docx",
"pptx",
"html",
"image",
"pdf",
"asciidoc",
"md",
"csv",
"xlsx",
"xml_uspto",
"xml_jats",
"json_docling",
],
"to_formats": ["md"],
"image_export_mode": "placeholder",
"do_ocr": True,
"force_ocr": False,
"ocr_engine": "easyocr",
"ocr_lang": None,
"pdf_backend": "dlparse_v2",
"table_mode": "accurate",
"abort_on_error": False,
"return_as_file": False,
"do_table_structure": True,
"include_images": True,
"images_scale": 2.0,
}
Contributor guide
Research direction
Start by exercising the /v1alpha/convert/source and /v1alpha/convert/file endpoints with repeated PDF requests using GPU, OCR, easyocr, and the listed parameters. Monitor nvidia-smi to determine whether memory keeps growing and whether OOM occurs after several documents. Done means GPU memory plateaus after model loading and repeated processing no longer causes OOM in the reported setup.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- api, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100