libwww-perl / libwww-perl/HTML-Parser
Feature request: Strict mode for decode_entities()
Nobody has claimed this yet.
- Dominant language
- Perl
- Stars
- 7
- Forks
- 15
- Avg merge
- 12h 8m
- Merged PRs (30d)
- 2
Description
Since 2004 decode_entities() supports the merging of surrogate pairs. See http://rt.cpan.org/Ticket/Display.html?id=7785 . This means that for example �� will be decoded into a single code point. My understanding that this not covered in any spec.
I therefore propose to add a function decode_entities_strict() that does the same as decode_entities() but rejects surrogate pairs.
Attached is a sample script that shows the effect-
surrogate_pair.pl.txt
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the existing decode_entities() entry point and run the attached surrogate_pair.pl.txt sample to observe the current surrogate-pair behavior. Add the proposed decode_entities_strict() behavior so surrogate pairs are rejected while the existing function remains unchanged, then verify the sample case and related decoding behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- perl
- Domain
- web-dev
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100