agentscope-ai / agentscope-ai/agentscope
feat(realtime): add cascaded realtime model (asr-llm-tts)
- Linguagem predominante
- Python
- Estrelas
- 31.6k
- Forks
- 3.5k
- Merge médio
- 1d 16h
- PRs com merge (30d)
- 103
Descrição
**Background**
The current `RealtimeModel` relies on native speech-to-speech APIs, which have limited provider coverage and controllability. A cascaded pipeline (ASR → LLM → TTS) is a natural alternative — more flexible, provider-agnostic, and easier to debug.
The proposed approach is to implement a `CascadedRealtimeModel` that shares the same interface as the existing `RealtimeModel`, so that `RealtimeAgent` can switch between speech-to-speech and cascaded mode purely by swapping the `model` field. This design assumption needs to be validated against the current `RealtimeModel` interface before implementation.
> Depends on #2564 (ASR module abstraction) to be merged first.
**Changes**
- Implement `CascadedRealtimeModel` compatible with the existing `RealtimeModel` interface
- `RealtimeAgent` switches between speech-to-speech and cascaded mode via the `model` field only — no agent-level changes required
- Support streaming across all three stages to minimize end-to-end latency
Guia de contribuição
Direção de pesquisa
Start with the `RealtimeModel` and `RealtimeAgent` definitions to confirm their current contract and how the `model` field is consumed. Validate the existing streaming path in the realtime flow and check issue #2564’s ASR abstraction context before drafting the new pipeline. Implement `CascadedRealtimeModel` as a drop-in model with the same interface so mode selection is only via `model`, and verify streaming behavior across ASR→LLM→TTS stays intact without RealtimeAgent code changes.
Escrita pelo modelo de indexação a partir do texto da issue.
Avaliação
- Stack de tecnologia
- python
- Domínio
- machine-learning
- Tipo de issue
- Funcionalidade
- Dificuldade
- 4/5
- Tempo estimado
- 3-5 dias
- Status de atividade
- Ativa
- Clareza
- Razoavelmente clara
- Facilidade para iniciantes
- 44/100