dead8309 / dead8309/markitdown-ts

XlsxConverter causes V8 heap OOM on spreadsheets with large cell ranges

Open
#17 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
HTML
Stars
142
Forks
13
PR merge metrics
No merged PRs in 30d

Description

## Description

`XlsxConverter` causes a fatal **V8 heap out of memory crash** when processing Excel files that have large cell ranges (e.g., `A1:I1048561`), even if the actual data only occupies a few rows.

This is related to #12 (empty rows in output), but the impact is much more severe: the entire Node.js process is killed.

## Root Cause

The conversion pipeline in `XlsxConverter` is:

1. `XLSX.utils.sheet_to_html(sheet)` — generates HTML for **every cell in the range**, including empty ones
2. `HtmlConverter._convert(htmlContent)` — parses the HTML into a DOM tree via Turndown/JSDOM

For a spreadsheet with range `A1:I1048561` (Excel's max row count, often caused by a stray formatting artifact), `sheet_to_html` generates **~248 MB of HTML**. When JSDOM builds a DOM tree from this, memory usage exceeds **4 GB**, hitting V8's heap limit.

## Reproduction

1. Create or use an Excel file where `sheet['!ref']` extends to a high row number (e.g., row 1,048,561) — this often happens when a cell far down has formatting but no data
2. Call `markitdown.convertBuffer(buffer, { file_extension: '.xlsx' })`
3. Node.js crashes with:

```
FATAL ERROR: Reached heap limit Allocation failed - JavaScript heap out of memory
```

## Test file details

- File size: **13 KB** (only ~20 rows of actual data)
- Sheet range reported by xlsx: `A1:I1048561`
- `sheet_to_html` output: **248 MB**
- `sheet_to_csv` output: **9 MB** (still large due to empty rows)
- Actual useful data: **~1.8 KB**

## Suggested Fix

Instead of converting via HTML → DOM → Markdown, use `XLSX.utils.sheet_to_json(sheet, { header: 1 })` and:

1. Filter out empty rows
2. Build Markdown tables directly from the data arrays

This avoids the HTML/DOM memory explosion entirely and also addresses #12.

```typescript
const rows = XLSX.utils.sheet_to_json(sheet, { header: 1, defval: '' });
const nonEmptyRows = rows.filter(row =>
row.some(cell => cell !== null && cell !== undefined && String(cell).trim() !== '')
);
// Build markdown table from nonEmptyRows...
```

## Environment

- markitdown-ts: 0.0.10
- Node.js: 22.22.0
- OS: macOS (Apple Silicon)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.