microsoft / microsoft/markitdown
Excel conversion replaces literal NA/NULL strings and blank cells with NaN
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Excel cells containing literal strings such as NA, NULL, None, n/a, and nan are converted to NaN. Real blank cells also appear as NaN. This loses distinctions in spreadsheet data, for example a region code NA versus an empty region.
Reproduced on main at eb31b5c9453628def5e6758a27a8e3a87b4ab101:
import io
from openpyxl import Workbook
from markitdown import MarkItDown
workbook = Workbook()
workbook.active.append(["region", "status", "empty"])
workbook.active.append(["NA", "NULL", None])
stream = io.BytesIO()
workbook.save(stream)
stream.seek(0)
print(MarkItDown().convert_stream(stream, file_extension=".xlsx").markdown)
The data row is | NaN | NaN | NaN |; it should be | NA | NULL | |.
Both XLSX and XLS converters use pandas' default missing-value recognition, which is useful for analysis but changes literal document text. keep_default_na=False preserves these strings and leaves blank cells empty. The XLSX fallback that repairs legacy showZeroes attributes also needs the same option.
I reproduced the failure for XLS, XLSX, and repaired XLSX and prepared a focused fix with regression tests. No model or remote service is needed. AI assistance: OpenAI Codex.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the XLSX and XLS converter entry points and the XLSX fallback that repairs legacy showZeroes attributes; review how each reads cell values through pandas. Add focused regression coverage for literal NA, NULL, and blank cells across XLS, XLSX, and repaired XLSX, and confirm the converted Markdown preserves the strings while leaving blanks empty.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 78/100