google / google/emoji-segmenter
Classification issues
- Dominant language
- C
- Stars
- 70
- Forks
- 19
- PR merge metrics
- No merged PRs in 30d
Description
While working on an alternative emoji segmentation implementation I noticed three presentation classification issues in the Ragel grammar:
**Bare `Emoji_Modifier` misclassified as text**
A lone modifier has `Emoji_Presentation=Yes` and should be classified as `is_emoji=true`. However, since `EMOJI_MODIFIER` is not included in `any_emoji`, it falls through to `text_run` and is classified as `is_emoji=false`.
**Bare `Emoji_Modifier_Base` misclassified as emoji**
[Nine codepoints](https://util.unicode.org/UnicodeJsps/list-unicodeset.jsp?a=%5B%3AEmoji_Modifier_Base%3A%5D%26%5B%3AEmoji_Presentation%3DNo%3A%5D) in the `EMOJI_MODIFIER_BASE` category have `Emoji_Presentation=No`, because `EMOJI_MODIFIER_BASE` is listed directly in `emoji_presentation`, a bare occurrence is classified as `is_emoji=true` regardless of the codepoint's default presentation property.
**Bare `Regional_Indicator` misclassified as text**
A lone regional indicator has `Emoji_Presentation=Yes` and should be classified as `is_emoji=true`. However, since `emoji_presentation` only includes `emoji_flag_sequence` (a pair of regional indicators), a single regional indicator falls through to `text_run` and is incorrectly classified as `is_emoji=false`.
**Unqualified keycap sequence not recognised**
An unqualified keycap sequence (`KEYCAP_BASE` + `COMBINING_ENCLOSING_KEYCAP`, e.g. `U+0023 U+20E3`) is not matched by any emoji rule. `emoji_keycap_sequence` requires an intervening VS-15 or VS-16, so the two codepoints are scanned separately as individual `text_run` tokens. Unicode includes unqualified keycap sequences in `emoji-test.txt` and they appear in legacy content, so failing to recognise them as a single emoji sequence is likely incorrect.
Contributor guide
Assessment
This issue has not been assessed yet.