[Feature Request] Native NPU inference support for Text-to-Speech (TTS) models
Nobody has claimed this yet.
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 152
- Avg merge
- 4h 14m
- Merged PRs (30d)
- 11
Description
Is your feature request related to a problem? Please describe.
Currently, FastFlowLM provides highly efficient, low-power STT (Whisper) and LLM inference on AMD Ryzen AI NPUs. However, completing a fully local voice pipeline (STT → LLM → TTS) requires waking up the discrete GPU or high-power CPU to render the final audio. This breaks the sub-2W power efficiency goal for edge deployments.
Describe the solution you'd like
Add q4nx conversion recipes and runtime support for efficient TTS architectures. Specifically, Mistral's Voxtral TTS (4B) is an ideal candidate for SRAM and memory bandwidth limits on current XDNA2 architecture.
Describe alternatives you've considered
Delegating TTS generation to standard local runtimes (e.g., vllm-omni or llama.cpp) via CPU/discrete GPU, which increases power overhead.
Additional context
- Target Use Case: End-to-end, on-device video translation and voice assistants.
- Environment: Linux (Pop!_OS 24 Cosmic), standard CLI automation workflows.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by locating FastFlowLM's existing q4nx conversion recipes and runtime support for Whisper and LLM inference, then compare those paths with Voxtral TTS (4B) requirements on XDNA2. Done means documenting and implementing the required conversion and runtime support, with validation that TTS inference runs on the NPU within the stated edge-deployment goals.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- audio-video-rtc, machine-learning, performance
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100