microsoft / microsoft/markitdown
Incorrect parsing of Unicode smart quotes from `.docx` files
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Bug: Incorrect parsing of Unicode smart quotes from .docx files
When using MarkItDown to convert .docx files created by Microsoft Word (default settings, smart quotes enabled), Unicode characters such as:
- Apostrophes (
’U+2019) - Left double quotes (
“U+201C) - Right double quotes (
”U+201D)
are incorrectly parsed and appear in the Markdown output as corrupted characters like Æ, ô, ö.
Steps to Reproduce:
- Create a new
.docxin Word with smart quotes enabled (default setting). - Add text such as:
It’s important to “quote” text properly. - Run MarkItDown to convert the
.docxto.md. - Observe corrupted characters in the output.
Expected Behavior:
Smart punctuation should either:
- Be preserved correctly as Unicode characters, or
- Be flattened gracefully to ASCII equivalents (
'and").
Actual Behavior:
Corrupted non-ASCII characters appear in Markdown.
Workarounds:
- Disabling smart quotes in Word avoids the issue.
- Alternative tools like Pandoc handle
.docxsmart punctuation correctly.
Environment:
- MarkItDown version: 0.1.1
- Python version: 3.12
- OS: Windows 11
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the issue with a Microsoft Word .docx containing the listed smart punctuation, then trace the MarkItDown .docx conversion entry point and its text decoding path. Done means the Markdown output preserves the Unicode characters or converts them cleanly to ASCII equivalents without corrupted characters.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100