microsoft / microsoft/markitdown
RSS/Atom: urljoin collapses significant repeated slashes in resolved links
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 186k
- Forks
- 13.7k
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 49
Description
Summary
RSS/Atom link resolution can collapse significant repeated slashes in URL paths. For example, exports/2026//report.csv resolves to a URL ending in exports/2026/report.csv, potentially identifying a different resource.
This is a longstanding upstream limitation in Python's standard-library urllib.parse.urljoin(), newly exposed by the feed link-resolution changes in #2432. MarkItDown v0.1.7 preserved the original relative reference.
Disposition: deferred; not a blocker for the upcoming release. This issue tracks the limitation and the upstream fix rather than requesting custom URL-resolution code immediately.
Reproduction
No network access is needed. Reproduced on Python 3.12.3:
from io import BytesIO
from markitdown import MarkItDown, StreamInfo
feed = b'''<rss version="2.0"><channel>
<title>Exports</title><link>https://example.com/bucket/</link>
<description>Available reports</description>
<item><title>Report</title><description><![CDATA[
<a href="exports/2026//report.csv">Download</a>
]]></description></item>
</channel></rss>'''
result = MarkItDown().convert_stream(
BytesIO(feed),
stream_info=StreamInfo(
extension=".rss",
charset="utf-8",
url="https://example.com/bucket/feed.xml",
),
)
print(result.markdown)
Actual link:
[Download](https://example.com/bucket/exports/2026/report.csv)
Expected link:
[Download](https://example.com/bucket/exports/2026//report.csv)
Cause and impact
_resolve_url() in packages/markitdown/src/markitdown/converters/_rss_converter.py calls urllib.parse.urljoin() directly. Its relative-path merge filters out empty interior segments.
Repeated slashes represent empty path segments and are not universally equivalent to a single slash. Some servers normalize them, but path-sensitive services and object stores can distinguish them. The same helper handles image references and xml:base; repeated slashes inherited from the base path can also be affected.
Upstream tracking
- Python issue: python/cpython#84774
- Proposed upstream fix: python/cpython#126679
As checked on September 11, 2026, both remain open and the filtering is still present in CPython's 3.13, 3.14, 3.15, and development main branches, as well as the local Python 3.12.3 runtime.
Because MarkItDown calls the standard library directly, it would inherit a future upstream fix when running on a Python release containing it. Updating MarkItDown alone would not change the behavior of an older Python runtime.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with packages/markitdown/src/markitdown/converters/_rss_converter.py and inspect _resolve_url(), then reproduce the behavior with the supplied Python snippet. Review python/cpython#84774 and python/cpython#126679; done requires an upstream resolution and verifying that MarkItDown preserves repeated path slashes on supported Python versions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100