graykode / graykode/nlp-tutorial
Some problems about Bert
Open
- Dominant language
- Jupyter Notebook
- Stars
- 14.9k
- Forks
- 3.9k
- PR merge metrics
- No merged PRs in 30d
Description
line 70: **index = randint(0, vocab_size - 1) # random index in vocabulary.**
I think the replace index can't involve 'cls' ,'sep' and 'mask'!
Contributor guide
Research direction
Read the BERT example around line 70 and verify how randint selects replacement tokens. Done means the replacement index cannot select cls, sep, or mask, with the relevant notebook behavior checked.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- jupyter-notebook, python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100