The preview of the new Python 3 port has broken HTML escaping in the XML feeds
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 144
- Forks
- 197
- PR merge metrics
- No merged PRs in 30d
Description
I am using:
O.S: Fedora 40
Browser: Firefox 131.0.2
Platform: desktop
Problem
The preview of the new Python 3 port has broken HTML escaping in the XML feeds
eg try to view this in the browser:
https://planetpython.org/3/rss10.xml
and it will complain about undefined entities, due to having raw unescaped HTML in the XML document
By comparison the original Python 2 code escaped HTML in the feed
$ wget https://planetpython.org/rss10.xml
$ grep "content:encoded" rss10.xml | head -1
<content:encoded><p>As is probably apparent from the sequence of blog posts about the topic in the
$ wget https://planetpython.org/3/rss10.xml
$ grep "content:encoded" rss10.xml.1 | head -1
<content:encoded><p>As is probably apparent from the sequence of blog posts about the topic in the
Details
This problem is caused by a mistake in the python 3 conversion done in #577, specially in commit https://github.com/python/planet/pull/577/commits/86e31f90403c4659471396beeba922584e08d12e replaced code patterns like:
feed[key] = sanitize.HTML(feed[key])
with
feed[key] = Markup(feed[key])
which is not providing functionally equivalent behaviour.
The sanitize.HTML method would parse the HTML and strip out various undesirable elements and attributes, and escaping was later performed by the template processor.
The Markup method will not parse anything, it'll just wrap the str in a Markup class, as a way to designate it as being safe to use as-is without further escaping. As a result when you later try to escape the variable in jinga using ... | e, it will do nothing at all, resulting in raw HTML being put into the XML document, leading to the later parsing errors.
I think either the original sanitizer code needs to be re-instated and made to work with py3, or perhaps an external library such as https://github.com/matthiask/html-sanitizer/ could be leveraged ?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the Python 3 feed generation affected by commit 86e31f90403c4659471396beeba922584e08d12e in #577, comparing the former sanitize.HTML call with Markup. Verify the fix against https://planetpython.org/3/rss10.xml and confirm content:encoded HTML is escaped so the XML parses without undefined-entity errors.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100