docling-project / docling-project/docling

High RAM consumption when converting large XLSX files (300k rows, 7 columns)

Open
#3,328 4 comments 0 reactions 1 assignee Claimed by @himax12 View on GitHub
bug xlsx
Dominant language
Python
Stars
66.4k
Forks
4.8k
Avg merge
2d 21h
Merged PRs (30d)
84

Description

## Description

Processing a large Excel file (~300,000 rows × 7 columns, ~32 MB on disk) causes extreme RAM consumption — peaking at **37–60 GB** on a 63 GB server — making the service effectively unusable for large spreadsheets.

## Environment

- **docling version**: (run `pip show docling` in your env)
- **docling-core version**: (run `pip show docling-core`)
- **Python**: 3.11
- **OS**: Ubuntu (Docker container, `python:3.11-slim` base)
- **Deployment**: FastAPI service calling `DocumentConverter.convert()`

## Steps to Reproduce

1. Take any `.xlsx` file with ~300,000 rows and 7 columns (~32 MB on disk)
2. Call `DocumentConverter.convert(path)` on it
3. Monitor RAM via `top` or `docker stats`

## Observed Behavior

- Conversion completes in ~116 seconds
- RAM spikes from ~22 GB baseline to **37–60 GB** during conversion
- Process PID shows **99–100% CPU** and **49–60% of 63 GB RAM** consumed
- Memory is not fully released after conversion

## Expected Behavior

RAM usage should scale reasonably with file size (~32 MB on disk). A 32 MB xlsx should not require 30–40 GB of RAM to process.

## Root Cause (suspected)

Docling's MsExcelDocumentBackend appears to load the entire workbook eagerly
via openpyxl and maps every cell to a TableCell object in the in-memory document
model. For 300k rows × 7 columns = 2.1 million cells, this creates an enormous
object graph that exhausts available RAM before the result is even returned.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.