docling-project / docling-project/docling

High memory usage when processing large PDFs with DoclingLoader (memory not released after each file)

Open
#2,829 2 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
3d 4h
Merged PRs (30d)
98

Description

I am using **DoclingLoader in Python** to extract content from a large number of PDF files (around **2000+ pages in total**). Each PDF is passed to the loader individually using its full file path.

During the extraction process, **memory consumption increases significantly and does not seem to be released after a PDF is processed**.

* Python process memory usage goes up to ~**12 GB**
* VS Code uses ~**1 GB**
* Total available RAM on the machine: **32 GB**

This makes it difficult to process large datasets and long-running jobs.

---

### Steps to Reproduce

1. Initialize `DoclingLoader`
2. Loop over multiple large PDF files (passing full path each time)
3. Extract content from each PDF
4. Observe memory usage over time

---

### Actual Behavior

* Memory usage continuously increases during processing
* Memory is not freed after a PDF is successfully processed
* Eventually leads to very high RAM usage or OOM issues

---

### Expected Behavior

* Memory should be released after each PDF is processed
* Overall memory usage should remain stable or grow minimally
* Ability to process large numbers of PDFs without exhausting system RAM

---

### Environment

* OS: Windows
* Python version: 3.11
* Docling version: docling 2.66.0
* RAM: 32 GB

---

### Questions

1. Is there a recommended way to **reduce memory consumption** when processing large PDFs?
2. Does Docling support **streaming, batching, or incremental processing**?
3. Is there a way to explicitly **release memory** after each PDF (e.g., cleanup hooks, context managers, or configuration options)?
4. Are there known memory leaks or best practices for large-scale PDF processing?

---

### Error

OutOfMemoryError when processing large PDFs with DoclingLoader (memory not released) ?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.