Differentiate bogus and normal comments in HTMLParser
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 77.2k
- Forks
- 35.9k
- PR merge metrics
- PR metrics pending
Description
The HTML specs support three kinds of bogus comments:
<?...>.HTMLParsercallshandle_pi()for it.<!...>.HTMLParserused to callunknown_decl()for it, but now (after #9295) it callshandle_comment().</...>if no ASCII letter follows/.HTMLParsercallshandle_comment()for it.
And, of course, handle_comment() is called for normal comments <!--...--> and <!--...--!>. This includes abnormal cases <!--> and <!---> which are treated as empty comment <!---->.
It is now impossible to differentiate <![if !(IE)]> from </[if !(IE)]> and <!--[if !(IE)]-->. This may be important, even if they are the same comment from the point of view of the HTML specs.
It was proposed in #70197 to add a new handler handle_bogus_comment() which calls handle_comment() by default to differentiate bogus comments from normal comments. Additional information should be passed to it besides the comment value to differentiate different kinds of bogus comments. For example, the character preceding the comment value (?, ! or /). But since handle_pi() is already called for <?...> and unknown_decl() used to be called for <!...>, we can just restore the use of unknown_decl() and add a new handler for </...>.
The second way will partially revert #9295. The difference is that a bogus comment (unknown declaration) starting with <![ will be terminated by first > instead of ]> or ]]>, in accordance to the HTML specs.
The problem is that unknown_decl() is also called for valid CDATA section (and trailing ]] is omitted). According to the HTML specs, its content should be treated as normal text, so we could simply call handle_data() (as for resolved character references), but for flexibility we can call a special method.
cc @ezio-melotti
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with HTMLParser and its handle_pi(), handle_comment(), unknown_decl(), and handle_data() entry points, then review the alternatives discussed in #9295 and #70197. Done requires an agreed API and behavior that distinguishes normal, bogus, and CDATA comments without breaking existing handlers.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- web-dev
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100