keymanapp / keymanapp/keyman

change(web): dictionary-based wordbreaker should take flag to enforce boundary at string's end 🔬

Open
#14,753 0 comments 0 reactions 1 assignee Claimed by @jahorton View on GitHub
change epic-dict-breaker web/
Dominant language
Pascal
Stars
534
Forks
143
Avg merge
2d 10h
Merged PRs (30d)
113

Description

https://github.com/keymanapp/keyman/blob/3e50a03ea0cf49b0421f97bf34812d2e9f7d7b29/web/src/engine/predictive-text/wordbreakers/src/main/dict/index.ts#L87-L92

In some of my recent work on epic/auto-correct, keeping in mind the needs of epic/dict-breaker, it appears we'll need to address this scenario in order to properly handle changes as the context window slides backward from backspaces and similar. It's more efficient to retokenize just part of the context, rather than the full window, and we don't want a partial token by mistake when doing related updates.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.