RTX3080+WSL成功运行模型量化部分代码，错误总结如下。
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 12.3k
- Forks
- 1.1k
- PR merge metrics
- No merged PRs in 30d
Description
1、首先量化模型用到triton库，所以只能在linux环境推理，我用的是wsl，装了ubuntu22.04。直接pip install triton也会报错，默认安装的版本带了post1后缀，运行后面的代码也会报错，所以安装时指定triton版本pip install triton==2.0.0 。
2、解决triton问题之后，运行如果出现python.h相关问题的话，需要执行安装sudo apt-get install python3.8-dev 。
3、另外试过更改moss_cli_demo.py 和oss_gui_demo.py的模型引用地址，都会报错，错误为缺少index.json文件，非量化模型是有不同的checkpoint的，所以文件夹会比量化模型多一个索引文件，这个问题待解决。
把目前记得的问题先记录下来，如果有相同配置的朋友出现其他问题，可以留言交流。
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Reproduce the quantized-model inference setup in WSL with Ubuntu 22.04, Triton 2.0.0, and the noted Python development package. Inspect moss_cli_demo.py and oss_gui_demo.py, then compare their model references with the quantized model directory and its missing index.json. Done means both demos can load the quantized model without the reported index-file error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- linux, python, ubuntu
- Domain
- machine-learning, operating-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100