microsoft / microsoft/markitdown
Add reader mode for automatic article content extraction from web pages
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Problem
When converting web pages to markdown, HtmlConverter processes the entire <body>, including navigation bars, sidebars, footers, cookie banners, and other boilerplate. This produces noisy markdown that buries the actual content.
Site-specific converters like WikipediaConverter solve this by targeting known DOM elements (e.g., div#mw-content-text), but there's no generic solution for arbitrary web pages.
Proposed solution
Add a reader mode option that automatically extracts the main article content from a web page before converting to markdown — similar to how Firefox Reader View strips away clutter.
This would be:
- Opt-in: a
reader_modekwarg on the API and a--reader-modeCLI flag, so existing behavior is unchanged - Optional dependency: following the existing pattern for pptx, pdf, etc.
Example
md = MarkItDown()
# Without reader mode - full page with nav, sidebar, footer
result = md.convert("https://example.com/blog-post")
# With reader mode - just the article content
result = md.convert("https://example.com/blog-post", reader_mode=True)
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating HtmlConverter and the code that defines MarkItDown's convert API and CLI options. Review the existing optional-dependency integrations, then determine how reader-mode extraction should be wired. Done means an opt-in reader_mode API kwarg and --reader-mode flag extract article content while preserving existing behavior by default.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100