ROCm / ROCm/FastFlowLM

[Feature Request] Native NPU inference support for Text-to-Speech (TTS) models

Open
#457 4 comments 6 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

model request
Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

Is your feature request related to a problem? Please describe.
Currently, FastFlowLM provides highly efficient, low-power STT (Whisper) and LLM inference on AMD Ryzen AI NPUs. However, completing a fully local voice pipeline (STT → LLM → TTS) requires waking up the discrete GPU or high-power CPU to render the final audio. This breaks the sub-2W power efficiency goal for edge deployments.

Describe the solution you'd like
Add q4nx conversion recipes and runtime support for efficient TTS architectures. Specifically, Mistral's Voxtral TTS (4B) is an ideal candidate for SRAM and memory bandwidth limits on current XDNA2 architecture.

Describe alternatives you've considered
Delegating TTS generation to standard local runtimes (e.g., vllm-omni or llama.cpp) via CPU/discrete GPU, which increases power overhead.

Additional context

  • Target Use Case: End-to-end, on-device video translation and voice assistants.
  • Environment: Linux (Pop!_OS 24 Cosmic), standard CLI automation workflows.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating FastFlowLM's existing q4nx conversion recipes and runtime support for Whisper and LLM inference, then compare those paths with Voxtral TTS (4B) requirements on XDNA2. Done means documenting and implementing the required conversion and runtime support, with validation that TTS inference runs on the NPU within the stated edge-deployment goals.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, linux
Domain
audio-video-rtc, machine-learning, performance
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.