docling-project / docling-project/docling
High memory usage when processing large PDFs with DoclingLoader (memory not released after each file)
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 3d 4h
- Merged PRs (30d)
- 98
Description
I am using **DoclingLoader in Python** to extract content from a large number of PDF files (around **2000+ pages in total**). Each PDF is passed to the loader individually using its full file path.
During the extraction process, **memory consumption increases significantly and does not seem to be released after a PDF is processed**.
* Python process memory usage goes up to ~**12 GB**
* VS Code uses ~**1 GB**
* Total available RAM on the machine: **32 GB**
This makes it difficult to process large datasets and long-running jobs.
---
### Steps to Reproduce
1. Initialize `DoclingLoader`
2. Loop over multiple large PDF files (passing full path each time)
3. Extract content from each PDF
4. Observe memory usage over time
---
### Actual Behavior
* Memory usage continuously increases during processing
* Memory is not freed after a PDF is successfully processed
* Eventually leads to very high RAM usage or OOM issues
---
### Expected Behavior
* Memory should be released after each PDF is processed
* Overall memory usage should remain stable or grow minimally
* Ability to process large numbers of PDFs without exhausting system RAM
---
### Environment
* OS: Windows
* Python version: 3.11
* Docling version: docling 2.66.0
* RAM: 32 GB
---
### Questions
1. Is there a recommended way to **reduce memory consumption** when processing large PDFs?
2. Does Docling support **streaming, batching, or incremental processing**?
3. Is there a way to explicitly **release memory** after each PDF (e.g., cleanup hooks, context managers, or configuration options)?
4. Are there known memory leaks or best practices for large-scale PDF processing?
---
### Error
OutOfMemoryError when processing large PDFs with DoclingLoader (memory not released) ?
Contributor guide
Assessment
This issue has not been assessed yet.