Decoding errors on Windows when using GPU
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 39.5k
- Forks
- 4.8k
- PR merge metrics
- No merged PRs in 30d
Description
When executing the command“”python -m fastchat.serve.model_worker --model-path E:\model\fastchat\model_weights\vicuna-7b, an error message as shown in the picture appears. Although the encoding configuration of PyCharm has been changed to UTF-8, the same error still occurs. However, when adding "--load-8bit" or "--device cpu" to the command, it can be executed normally. Can anyone please provide some guidance?
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the python -m fastchat.serve.model_worker --model-path ... entry point and reproduce the failure on Windows with GPU enabled. Compare it with the working --load-8bit and --device cpu modes, and obtain the error text shown in the attached image. Done means the GPU mode runs without the decoding error or the supported limitation is documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100