huggingface / huggingface/course
Issue with Chapter 2 (Tokenizers/Tokenization) - output is different
- Dominant language
- MDX
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 13m
- Merged PRs (30d)
- 1
Description
Hey Hugging face,
I am opening this issue because I am currently following your course (that is great by the way) and noticed some difference between the output displayed in the tokenization section and the output in my notebook.
On your lesson it looks like this

When I am executing the same code in my EC2 instance on AWS I have that

with this list `['Using', 'a', 'Trans', '##former', 'network', 'is', 'simple']` .
I am presuming that the model `bert-base-cased` could have changed since the edition of the course so it really not a big deal but it's impact the conclusion here

I don't know if the issue is on my side so there is the details on my current setup if you want to try to reproduce it:
* AWS ec2 p3.2xlarge
* environment miniconda Latest
* dependencies (in the requirements.txt)
[requirements.txt](https://github.com/huggingface/course/files/8430024/requirements.txt)
Don't hesitate if you have any questions
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.