modelscope / modelscope/FunASR
没有vllm时使用fun-asr-nano每次都重新加载模型
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 20.4k
- Forks
- 2k
- Avg merge
- 4h 55m
- Merged PRs (30d)
- 169
Description
如题,在环境中没安装vllm时,若使用fun-asr-nano,则每次都会重新加载模型。
问题出现在_server_app.py的159~172行。
问题在于:
- vLLM 加载失败时 →
app.state.engine仍然是None - 降级到
AutoModel时 → 虽然模型被存入app.state.fallback_models["fun-asr-nano"],但没有设置任何标志阻止下次重新加载 - 下次请求时 → 因为
app.state.engine is None,条件不成立,再次进入_load_vllm_engine() - 再次失败 → 再次加载
AutoModel(重复步骤 2-3)
另外,第273~280行这个逻辑也不是很合理,运行起来的python应该不能动态地就多了vllm库,所以server跑起来检测vllm不存在后,使用一个状态变量记一下就行了,至少server重启之前这个变量的值应该都不会变了。然后再所有模型都可以走_load_fallback。对于有vllm,则判断模型是不是fun-asr-nano,再决定走不走_load_vllm_engine()。这样我觉得更合理一些。
以现有代码,最小改变是:
第160行之后加两行:
if app.state.fallback_models["fun-asr-nano"]:
return app.state.fallback_models["fun-asr-nano"]
供参考。
感谢您的付出。
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with _server_app.py lines 159-172 and trace how _load_vllm_engine and the fallback_models entry are used across requests. Review lines 273-280 as well, then make the fallback state prevent repeated loading when vLLM is unavailable; done means repeated fun-asr-nano requests reuse the loaded fallback model.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 76/100