microsoft / microsoft/foundry-local
[Feature Request] Add Kokoro-82M text-to-speech support
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 2.6k
- Forks
- 369
- Avg merge
- 2d 17h
- Merged PRs (30d)
- 39
Description
## Describe the feature request
Add text-to-speech (TTS) as a supported Foundry Local task and onboard **Kokoro-82M v1.0** into the curated model catalog.
Foundry Local currently exposes speech-to-text models such as Whisper, Nemotron ASR, and Parakeet, but no model that produces speech. Kokoro is a particularly good fit for on-device inference:
- 82 million parameters
- Apache-2.0-licensed model weights
- Existing ONNX Runtime conversions with fp32, fp16, and quantized variants
- Approximately 86-326 MB depending on quantization
- Existing CPU and WebGPU deployments
- Multiple voices and languages
Relevant sources:
- Official model card: https://huggingface.co/hexgrad/Kokoro-82M
- ONNX v1.0 conversion: https://huggingface.co/onnx-community/Kokoro-82M-v1.0-ONNX
- NVIDIA-optimized ONNX model: https://huggingface.co/nvidia/kokoro-82M-onnx-opt
- ONNX Runtime wrapper: https://github.com/thewh1teagle/kokoro-onnx
The catalog artifact would need to include or acquire the required tokenizer/phonemizer assets and voice embeddings in addition to the ONNX graph. The team should validate or reproduce the conversion rather than necessarily consuming a community artifact directly.
This also requires a supported TTS inference surface. A possible shape, without prescribing the design, could include:
- A `text-to-speech` task type
- An SDK speech-generation client/session
- An OpenAI-compatible `/v1/audio/speech` endpoint
- CLI support for generating a WAV file from text
- CPU and WebGPU variants across supported desktop platforms
## Describe scenario use case
Applications should be able to synthesize speech fully on-device using the same runtime and model-management experience already used for chat, embeddings, multimodal inference, and transcription.
Example scenarios include:
- Accessibility and read-aloud experiences
- Private/offline voice assistants
- Spoken notifications and summaries
- Low-latency conversational applications combining local STT, an LLM, and TTS
- Cross-platform desktop applications that should not require a second inference runtime
Today these applications must integrate a separate Kokoro runtime or use a cloud TTS service, losing Foundry Local's unified hardware selection, caching, packaging, and offline experience.
## Customer requirement and impact
TTS completes the local conversational loop: **speech input -> local reasoning -> speech output**. Kokoro offers a small, permissively licensed starting model that is practical for consumer devices and substantially lighter than generative TTS models in the billion-parameter range.
Adding it would provide immediate value to accessibility, assistant, productivity, and privacy-sensitive applications while establishing the runtime/API foundation for future speech-synthesis models.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Review the existing speech-to-text task and model-management experience first, then trace the proposed /v1/audio/speech endpoint and CLI generation flow. Done means defining and implementing a supported TTS surface, onboarding Kokoro-82M with its tokenizer, phonemizer, and voice assets, and validating CPU and WebGPU variants across supported desktop platforms.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp
- Domain
- api, cli, machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100