fohrloop / fohrloop/granite-tools
Calculate unigram and repeats scores purely based on bigram data
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Motivation:
- When ordering and comparing ngrams (bigrams and unigrams) there is more comparisons to be made if unigrams are included (minor pain)
- Comparing unigram to bigram in bigram score ratio file is difficult (major pain)
- The unigram to bigram score ratios probably are more inaccurate and cause the bigram model fit to be poorer (at least removing score ratios including "Q" improved a fit when I tried)
- There's already some code that may do this thing (partially). It has been used to generate an [effort grid](https://www.reddit.com/r/KeyboardLayouts/comments/1i8t0x8/effort_grid_for_36_key_layout_based_on_bigram/) for all the unigrams.
- Unigram scores given by human manually is probably not so accurate as the automatically calculated unigram scores, which tell how "good" certain key locations are _within bigrams_ (on average). So it probably more accurately describes the score of the unigrams in how they're used (as part of bigrams or part of longer text; not isolated keys).
Risks and Cons:
- I have written "bigrams (and unigrams)" in million places. Probably will not find all of them.
- The exact logic is TBD. I have to experiment on this.
- This most likely requires at least some type of new parameter to be included in the granite config (but that's still easier than including unigrams in the manual process steps)
Idea:
- Remove unigrams from all steps requiring human interaction. Calculate the unigram scores from the bigram scores.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.