OpenBMB / OpenBMB/MiniCPM-o-Demo
有考虑做快慢思考的系统吗？
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 386
- Forks
- 81
- Avg merge
- 1h 59m
- Merged PRs (30d)
- 3
Description
你好，首先感谢开源这样一个优秀的全双工语音模型与系统方案。
我们之前都是用asr+llm+tts的级联方案，整体来说，时延上还是没有咱们的好。
但在实际生产上使用的话，我目前遇到了两个问题：
- 这是一个端到端的系统，无法嵌入已有的业务workflow（例如：rag召回，mcp订单查询，固定话术反问等等）
- 模型size较小（~9B），对于复杂场景的回复，还是不如大尺寸的模型
我这两天在尝试自己手搓一个快慢系统的方案，即：
- 快系统要求回复迅速，快速承接用户问题，可以只回复一句话、垫词等。这里仍然是使用咱们minicpm原有的方案。
- 慢系统：仍然使用传统的asr+llm+tts的级联方案，保持不变，tts相关音频直接拼接在快系统音频的后面，保证回复的质量
快慢系统用一个统一的调度器进行调度，打断通过minicpm中的listen与speak的轮转来控制
这里是想问问咱们官方有考虑后续出类似的集成方案吗？（许愿
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
The issue does not name implementation files, tests, or a concrete entry point. Start by reviewing the existing listen/speak turn-taking flow and the ASR, LLM, and TTS integration points described here. Done would require a defined fast/slow orchestration design that supports workflow steps and preserves interruption behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- ai, audio-video-rtc, backend-api-design
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100