docling-project / docling-project/docling-serve
Issue: GPU Memory Leak in quay.io/docling-project/docling-serve-cu124 Image (docling-serve container image)
- Dominant language
- Python
- Stars
- 1.8k
- Forks
- 344
- Avg merge
- 6d 11h
- Merged PRs (30d)
- 8
Description
**Description:**
I’m encountering a potential GPU memory leak when using the Docker image `quay.io/docling-project/docling-serve-cu124 ` for Docling Serve on a machine equipped with an NVIDIA A100-SXM4-40GB GPU and CUDA version 12.9.
**Issue Details:**
- After uploading an initial batch of 10 files, the GPU memory usage increased.
- Uploading a second batch of 20 files caused a further increase in memory usage.
- After uploading 100 files, the GPU memory usage continued to increase and eventually reached high utilization.
- Critically, even after all conversions completed, the GPU memory did not release and remained fully occupied.
I have attached a GPU usage screenshot for reference.

**Expected Behavior:**
The GPU memory should be released after each conversion task is completed, or at least gradually cleaned up to allow continuous processing without exhausting memory resources.
**Request:**
Could you please provide guidance on:
- How to properly manage or release GPU memory after each job?
- Whether this is a known issue in this image version or a bug in memory handling?
- Any recommended patches, updates, or settings (e.g., Torch/transformer memory cleanup, torch.cuda.empty_cache(), etc.)?
Thank you.
Contributor guide
Research direction
Reproduce the behavior with the quay.io/docling-project/docling-serve-cu124 image on the stated A100-SXM4-40GB and CUDA 12.9 setup, processing batches of 10, 20, and 100 files while monitoring GPU memory. Determine whether memory remains occupied after conversions finish; done means the cause and a supported cleanup, update, patch, or setting are identified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- docker, python
- Domain
- api, backend, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100