feat(web): Add Support for Simple Misspelling Aliases to lexical models
- Dominant language
- Pascal
- Stars
- 534
- Forks
- 143
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 113
Description
While the Unilex project is a valuable resource of real-world, practical, lexicons, it is also a database of misspelled words and their frequencies. Unfortunately, correctly spelled and misspelled words are not marked and so are indistinguishable in their datasets. The task of eliminating misspelled words, and adding their frequency counts to the correctly spelled word, is left to the Keyman Lexical Model maintainer. Which is fine.
I propose here that the lexical model, along with Keyman's predictive text features, support a soft correction of misspelled words. Thus, when a misspelled word is typed by a user, the correctly spelled word will be offered for selection. I believe this can be done with a data-driven approach thus avoiding the need to add language-specific spelling logic to the predictive text engine. The simplest approach that I can think of is to list the misspelled words after the 2nd column of the lexicon `.tsv` files. For example:
```
correctWord 54321 badSpelling1 badSpelling2 badSpelling3 ...
```
Beginning from the 3rd column, all listed words are the common misspellings that are "aliased" to the correct spelling. So for example, when a user types partly or fully `badSpelling3` the predictive text feature will offer in its place the correctly spelled `correctWord`, and apply the same frequency weight to the bad spelling as the correct spelling (by virtue of being an "alias").
Again this is a generic approach that avoids language-specific logic, the lexical model maintainer can use Unilex data and other resources to create the list of incorrect spellings.
Alternatively, the misspelling aliases could reside in a companion file that is referenced from the `.ts` file and loaded at build time.
Contributor guide
Assessment
This issue has not been assessed yet.