emberjs / emberjs/simple-html-tokenizer
EntityParser can't handle encoded emoji
- Dominant language
- TypeScript
- Stars
- 85
- Forks
- 34
- PR merge metrics
- No merged PRs in 30d
Description
While the tokenizer will gracefully decode most encoded characters:
```
⇒ node
> var Tokenizer = require( 'simple-html-tokenizer' );
undefined
> Tokenizer.tokenize( '&' )[ 0 ].chars === '&'
true
```
It doesn't handle characters whose encodings exceed 16 bits (e.g. emoji):
```
⇒ node
> var Tokenizer = require( 'simple-html-tokenizer' );
undefined
> Tokenizer.tokenize( '😅' )[ 0 ].chars === '😅'
false
```
It may be that `EntityParser` should use `String.fromCodePoint` in place of `String.fromCharCode` instead, or an equivalent polyfill?
Related:
- https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/String/fromCodePoint
- https://thekevinscott.com/emojis-in-javascript/
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.