modelscope / modelscope/FunASR

没有vllm时使用fun-asr-nano每次都重新加载模型

Open Beginner friendly
#3,401 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug needs feedback
Dominant language
Python
Stars
20.4k
Forks
2k
Avg merge
4h 55m
Merged PRs (30d)
169

Description

如题,在环境中没安装vllm时,若使用fun-asr-nano,则每次都会重新加载模型。

问题出现在_server_app.py的159~172行。
问题在于:

  • vLLM 加载失败时 → app.state.engine 仍然是 None
  • 降级到 AutoModel 时 → 虽然模型被存入 app.state.fallback_models["fun-asr-nano"],但没有设置任何标志阻止下次重新加载
  • 下次请求时 → 因为 app.state.engine is None,条件不成立,再次进入 _load_vllm_engine()
  • 再次失败 → 再次加载 AutoModel(重复步骤 2-3)

另外,第273~280行这个逻辑也不是很合理,运行起来的python应该不能动态地就多了vllm库,所以server跑起来检测vllm不存在后,使用一个状态变量记一下就行了,至少server重启之前这个变量的值应该都不会变了。然后再所有模型都可以走_load_fallback。对于有vllm,则判断模型是不是fun-asr-nano,再决定走不走_load_vllm_engine()。这样我觉得更合理一些。

以现有代码,最小改变是:
第160行之后加两行:

if app.state.fallback_models["fun-asr-nano"]:
    return app.state.fallback_models["fun-asr-nano"]

供参考。
感谢您的付出。

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with _server_app.py lines 159-172 and trace how _load_vllm_engine and the fallback_models entry are used across requests. Review lines 273-280 as well, then make the fallback state prevent repeated loading when vLLM is unavailable; done means repeated fun-asr-nano requests reuse the loaded fallback model.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
backend
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.