docling-project / docling-project/docling
High RAM consumption when converting large XLSX files (300k rows, 7 columns)
- Dominant language
- Python
- Stars
- 66.4k
- Forks
- 4.8k
- Avg merge
- 2d 21h
- Merged PRs (30d)
- 84
Description
## Description
Processing a large Excel file (~300,000 rows × 7 columns, ~32 MB on disk) causes extreme RAM consumption — peaking at **37–60 GB** on a 63 GB server — making the service effectively unusable for large spreadsheets.
## Environment
- **docling version**: (run `pip show docling` in your env)
- **docling-core version**: (run `pip show docling-core`)
- **Python**: 3.11
- **OS**: Ubuntu (Docker container, `python:3.11-slim` base)
- **Deployment**: FastAPI service calling `DocumentConverter.convert()`
## Steps to Reproduce
1. Take any `.xlsx` file with ~300,000 rows and 7 columns (~32 MB on disk)
2. Call `DocumentConverter.convert(path)` on it
3. Monitor RAM via `top` or `docker stats`
## Observed Behavior
- Conversion completes in ~116 seconds
- RAM spikes from ~22 GB baseline to **37–60 GB** during conversion
- Process PID shows **99–100% CPU** and **49–60% of 63 GB RAM** consumed
- Memory is not fully released after conversion
## Expected Behavior
RAM usage should scale reasonably with file size (~32 MB on disk). A 32 MB xlsx should not require 30–40 GB of RAM to process.
## Root Cause (suspected)
Docling's MsExcelDocumentBackend appears to load the entire workbook eagerly
via openpyxl and maps every cell to a TableCell object in the in-memory document
model. For 300k rows × 7 columns = 2.1 million cells, this creates an enormous
object graph that exhausts available RAM before the result is even returned.
Contributor guide
Assessment
This issue has not been assessed yet.