microsoft / microsoft/markitdown
Potential Optimization of `xlsx` Converter Using `pandas` `to_markdown` Method
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Hello,
I have observed that the current implementation of the xlsx converter processes sheets by converting them to HTML and then parsing that HTML to Markdown:
for s in sheets:
md_content += f"## {s}\n"
html_content = sheets[s].to_html(index=False)
md_content += (
self._html_converter.convert_string(
html_content, **kwargs
).markdown.strip()
+ "\n\n"
)
I am wondering if this approach is necessary or if it could be optimized by directly using pandas' to_markdown method:
for s in sheets:
md_content += f"## {s}\n"
sheet_md_content = sheets[s].to_markdown(index=False)
md_content += (
sheet_md_content.strip()
+ "\n\n"
)
- Is there a specific reason why the implementation relies on converting to HTML and then parsing back to Markdown? For example, does the HTML conversion preserve certain formatting or features that
to_markdowndoes not handle? - Are there any limitations or edge cases where
to_markdownmight not be sufficient or could produce different results? - Would switching to
to_markdownimpact the fidelity or formatting of the output?
Could you please clarify if the current approach is intentional due to specific requirements? If not, would it be possible to consider this optimization to improve performance ?
Thank you for your insights.
Best regards
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the xlsx converter loop shown in the issue and compare its HTML-to-Markdown path with pandas DataFrame.to_markdown(index=False). Investigate whether the two approaches differ for formatting or edge cases, then determine whether the proposed replacement preserves output fidelity and improves performance. Done means the approach is supported by evidence and any resulting behavior is validated.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- pandas, python
- Domain
- data
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100