deepspeedai / deepspeedai/DeepSpeedExamples
ascii' codec can't decode byte
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 6.8k
- Forks
- 1.1k
- Avg merge
- 2d 16h
- Merged PRs (30d)
- 1
Description
When I use Chinese corpus，I got an error
UnicodeDecodeError:'ascii' codec can't decode byte 0xe7 in position 11:oridinal not in range(128)
but I can't find the bug
Does anybody know why？
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the UnicodeDecodeError with the Chinese corpus, then trace where the corpus is read in the relevant example. Identify the source of the ASCII decoding assumption and verify that the issue no longer occurs with non-ASCII input.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100