agentscope-ai / agentscope-ai/agentscope
feat(tts): support voice cloning and local TTS
- Vorherrschende Sprache
- Python
- Sterne
- 31.5k
- Forks
- 3.5k
- Ø Merge
- 1 T. 23 Std.
- Gemergte PRs (30 T.)
- 95
Beschreibung
## Background
AgentScope already provides a unified `TTSModelBase` abstraction and supports multiple API-based TTS providers. However, two important capabilities are still missing:
1. An end-to-end API-based voice cloning workflow that creates and manages custom voices from reference audio.
2. Local TTS inference for offline and privacy-sensitive deployments, including local voice cloning.
This issue tracks the overall implementation of voice cloning and local TTS support in AgentScope.
## Scope
- Support API-based voice cloning:
- Create a custom voice from reference audio.
- Manage provider-side custom voices where supported.
- Use the returned voice ID directly with existing TTS models.
- Support local TTS inference:
- Provide a lightweight preset-voice engine.
- Provide at least one zero-shot local voice-cloning engine.
- Support CPU, CUDA, and MPS where supported by the upstream engine.
- Expose consistent capability metadata, such as `voice_cloning`, through TTS model cards.
- Keep local model dependencies optional and lazily loaded.
- Add unit tests, documentation, and runnable examples.
- Document privacy, consent, and responsible-use requirements for reference audio.
## Out of Scope
- STT or full realtime speech-to-speech interaction.
- Training or fine-tuning a new speaker model.
- Hosting or distributing cloned voices as a standalone service.
## Definition of Done
- Both sub-issues are completed.
- API and local voices can be selected through the existing TTS configuration flow.
- Voice-cloning capabilities are discoverable from model metadata.
- Existing TTS providers and `TTSMiddleware` continue to work without regression.
- Documentation includes API-based and fully local examples.
Beitragsleitfaden
Bewertung
Dieses Issue wurde noch nicht bewertet.