docling-project / docling-project/docling
Potential memory leak in docling pdf conversion
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
### Bug
I've found memory use gradually growing with multiple successive conversions of simple PDF documents using the standard PDF conversion pipeline. I have observed this behaviour with/without OCR enabled and when switching to PyPdfium backend also.
I have also not found any way to free this memory, deleting and garbage collecting all references do not seem to clear it. Instantiating a new DocumentConverter also just increases the base memory load (I believe it must reload a lot of the heavy ML stuff in the background)
### Steps to reproduce
I can reproduce the issue with this simple script:
```python
from docling.document_converter import DocumentConverter
from pathlib import Path
import psutil
import os
def get_current_process_memory() -> float:
"""Prints current process memory usage."""
process = psutil.Process(os.getpid())
mem_mb = process.memory_info().rss / 1024 / 1024
print(f"Total Memory: {mem_mb:.2f} MB")
return mem_mb
def plot_memory_usage() -> None:
import pandas as pd
import matplotlib.pyplot as plt
df = pd.DataFrame(memory_usage, columns=["memory_usage"])
df.plot(kind="line")
# save plot to png
plt.savefig("memory_usage.png")
if __name__ == "__main__":
converter = DocumentConverter()
pdf = Path("lorem_ipsum.pdf")
memory_usage = []
for i in range(100):
doc = converter.convert(pdf)
memory_usage.append(get_current_process_memory())
plot_memory_usage()
```
This is using this simple 1 page sample PDF
[lorem_ipsum.pdf](https://github.com/user-attachments/files/24164895/lorem_ipsum.pdf)
And produces the following output graph of memory use (X axis: iterations, Y axis: memory (MB)):
### Docling version
Docling version: 2.64.0
Docling Core version: 2.54.0
Docling IBM Models version: 3.10.3
Docling Parse version: 4.7.2
Python: cpython-312 (3.12.12)
Platform: Windows-11-10.0.26100-SP0
### Python version
3.12.12
Contributor guide
Assessment
This issue has not been assessed yet.