docling-project / docling-project/docling-serve

Issue: GPU Memory Leak in quay.io/docling-project/docling-serve-cu124 Image (docling-serve container image)

Open
#233 6 comments 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
1.8k
Forks
344
Avg merge
6d 11h
Merged PRs (30d)
8

Description

**Description:**

I’m encountering a potential GPU memory leak when using the Docker image `quay.io/docling-project/docling-serve-cu124 ` for Docling Serve on a machine equipped with an NVIDIA A100-SXM4-40GB GPU and CUDA version 12.9.

**Issue Details:**

- After uploading an initial batch of 10 files, the GPU memory usage increased.
- Uploading a second batch of 20 files caused a further increase in memory usage.
- After uploading 100 files, the GPU memory usage continued to increase and eventually reached high utilization.
- Critically, even after all conversions completed, the GPU memory did not release and remained fully occupied.

I have attached a GPU usage screenshot for reference.

![Image](https://github.com/user-attachments/assets/3d33336d-d89e-40dc-88a6-5da5304c8f21)

**Expected Behavior:**

The GPU memory should be released after each conversion task is completed, or at least gradually cleaned up to allow continuous processing without exhausting memory resources.

**Request:**

Could you please provide guidance on:

- How to properly manage or release GPU memory after each job?
- Whether this is a known issue in this image version or a bug in memory handling?
- Any recommended patches, updates, or settings (e.g., Torch/transformer memory cleanup, torch.cuda.empty_cache(), etc.)?

Thank you.

Contributor guide

Open the contributing guide

Research direction

Reproduce the behavior with the quay.io/docling-project/docling-serve-cu124 image on the stated A100-SXM4-40GB and CUDA 12.9 setup, processing batches of 10, 20, and 100 files while monitoring GPU memory. Determine whether memory remains occupied after conversions finish; done means the cause and a supported cleanup, update, patch, or setting are identified.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
api, backend, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.