microsoft / microsoft/markitdown

Excel conversion replaces literal NA/NULL strings and blank cells with NaN

Open
#2,498 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
186k
Forks
13.7k
Avg merge
1d 4h
Merged PRs (30d)
49

Description

Excel cells containing literal strings such as NA, NULL, None, n/a, and nan are converted to NaN. Real blank cells also appear as NaN. This loses distinctions in spreadsheet data, for example a region code NA versus an empty region.

Reproduced on main at eb31b5c9453628def5e6758a27a8e3a87b4ab101:

import io
from openpyxl import Workbook
from markitdown import MarkItDown

workbook = Workbook()
workbook.active.append(["region", "status", "empty"])
workbook.active.append(["NA", "NULL", None])
stream = io.BytesIO()
workbook.save(stream)
stream.seek(0)
print(MarkItDown().convert_stream(stream, file_extension=".xlsx").markdown)

The data row is | NaN | NaN | NaN |; it should be | NA | NULL | |.

Both XLSX and XLS converters use pandas' default missing-value recognition, which is useful for analysis but changes literal document text. keep_default_na=False preserves these strings and leaves blank cells empty. The XLSX fallback that repairs legacy showZeroes attributes also needs the same option.

I reproduced the failure for XLS, XLSX, and repaired XLSX and prepared a focused fix with regression tests. No model or remote service is needed. AI assistance: OpenAI Codex.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at the XLSX and XLS converter entry points and the XLSX fallback that repairs legacy showZeroes attributes; review how each reads cell values through pandas. Add focused regression coverage for literal NA, NULL, and blank cells across XLS, XLSX, and repaired XLSX, and confirm the converted Markdown preserves the strings while leaving blanks empty.

Written by the indexing model from the issue text.

Assessment

Tech stack
pandas, python
Domain
tooling
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.