fohrloop / fohrloop/granite-tools
Make the bigram score scale fixed
- Dominant language
- Python
- Stars
- 1
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
One problem with fitting the bigram scores (using bigram score ratios and `granite-bigram-scores-fit`) is that sometimes the scale can vary greatly. The highest bigram score can become 6.0 or 12.0. Here is an example where I simply removed the unigram Q and score ratios associated to it from the bigram score ratio file (the highest score moves from about 7.8 to about 9.6):

As a fix, when calculating the bigram scores get the score ratio between the first and last ranked ngram in bigram score ratio file. For example (this is same as `ZQ/D` on the left side):
```yml
- ref: "K"
ngram: "/P"
score_ratio: 4.5
```
and scale all the calculated scores in a way that the resulting scores satisfy this.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.