chore(web): global input normalization in lexical models
- Dominant language
- Pascal
- Stars
- 534
- Forks
- 143
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 113
Description
> 5. Then, once we have (4) done, the model engine should inherit the normalization form of the associated keyboard -- so if the keyboard emits NFC, the model engine should normalize(nfc) its outputs. And the converse for NFD. The internals don't matter so much, but we use NFC throughout, so let's stick with that. Inputs to the model should be normalized to NFC (they probably already are?)
The lexical model compiler already does the normalization to NFC at build time, so I think we can declare this issue done. Remaining work is in referenced issues.
_Originally posted by @mcdurdin in https://github.com/keymanapp/keyman/issues/2880#issuecomment-1726770721_
If using a Trie-based wordlist, part of the model functionality converts both sides to NFD (with the default search-term keyer) when doing a word lookup. It's consistent and has been working well, so no worries here.
For custom models, and on the global level... we technically don't yet enforce a normalization pattern.
Contributor guide
Assessment
This issue has not been assessed yet.