推理报错CUDA:oom & 添加参数--use-bminf --memory-limit 10执行后，报错缺少参数
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 2.4k
- Forks
- 176
- PR merge metrics
- No merged PRs in 30d
Description
模型:2b的压缩模型
内存16G
显卡：v100 16G
部署后，运行python text_generation.py报错 CUDA：OOM
增加参数--use-bminf --memory-limit 10后执行，报错缺少参数
对bminf的版本是否有要求？具体要求什么版本？
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with text_generation.py and reproduce the CUDA OOM using the stated 2B compressed model, 16G system memory, and V100 16G GPU. Then inspect how --use-bminf and --memory-limit 10 are parsed and identify the missing parameter and any required bminf version. Done means the required invocation and compatibility requirements are documented or the reported failure is fixed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100