huggingface / huggingface/course

Potential calculation error on WordPiece section

Open
#500 0 comments 2 reactions 0 assignees View on GitHub
Dominant language
MDX
Stars
4.2k
Forks
1.4k
Avg merge
13m
Merged PRs (30d)
1

Description

Hello, I was reading the [WordPiece section](https://huggingface.co/course/chapter6/6?fw=pt) and found the score calculation between `hu` and `##g` from the end of ***training algorithm*** part doesn't make sense to me. Specifically, the frequency of `hug` should be 10 while both `hu` and `##g` have a frequency of 15, which yields a score of 10/(15*15), and is not equal to 1/15, as shown on the website.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.