dead8309 / dead8309/markitdown-ts
XlsxConverter causes V8 heap OOM on spreadsheets with large cell ranges
- Dominant language
- HTML
- Stars
- 142
- Forks
- 13
- PR merge metrics
- No merged PRs in 30d
Description
## Description
`XlsxConverter` causes a fatal **V8 heap out of memory crash** when processing Excel files that have large cell ranges (e.g., `A1:I1048561`), even if the actual data only occupies a few rows.
This is related to #12 (empty rows in output), but the impact is much more severe: the entire Node.js process is killed.
## Root Cause
The conversion pipeline in `XlsxConverter` is:
1. `XLSX.utils.sheet_to_html(sheet)` — generates HTML for **every cell in the range**, including empty ones
2. `HtmlConverter._convert(htmlContent)` — parses the HTML into a DOM tree via Turndown/JSDOM
For a spreadsheet with range `A1:I1048561` (Excel's max row count, often caused by a stray formatting artifact), `sheet_to_html` generates **~248 MB of HTML**. When JSDOM builds a DOM tree from this, memory usage exceeds **4 GB**, hitting V8's heap limit.
## Reproduction
1. Create or use an Excel file where `sheet['!ref']` extends to a high row number (e.g., row 1,048,561) — this often happens when a cell far down has formatting but no data
2. Call `markitdown.convertBuffer(buffer, { file_extension: '.xlsx' })`
3. Node.js crashes with:
```
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
```
## Test file details
- File size: **13 KB** (only ~20 rows of actual data)
- Sheet range reported by xlsx: `A1:I1048561`
- `sheet_to_html` output: **248 MB**
- `sheet_to_csv` output: **9 MB** (still large due to empty rows)
- Actual useful data: **~1.8 KB**
## Suggested Fix
Instead of converting via HTML → DOM → Markdown, use `XLSX.utils.sheet_to_json(sheet, { header: 1 })` and:
1. Filter out empty rows
2. Build Markdown tables directly from the data arrays
This avoids the HTML/DOM memory explosion entirely and also addresses #12.
```typescript
const rows = XLSX.utils.sheet_to_json(sheet, { header: 1, defval: '' });
const nonEmptyRows = rows.filter(row =>
row.some(cell => cell !== null && cell !== undefined && String(cell).trim() !== '')
);
// Build markdown table from nonEmptyRows...
```
## Environment
- markitdown-ts: 0.0.10
- Node.js: 22.22.0
- OS: macOS (Apple Silicon)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.