audio-cpp using a fish-audio doesn't work with referance audio/text
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 49.2k
- Forks
- 4.5k
- Avg merge
- 1d 3m
- Merged PRs (30d)
- 239
Description
LocalAI version:
localai/localai:master-gpu-hipblas
sha256:f3ff922777a09ec7f63093df7482ebb15d3f94c0f8611d49143fe455b4f7144d
Environment, CPU architecture, OS, and Version:
I'm doing this on unraid 7.3, using portainer.
AMD Ryzen 9 5900X
Radeon AI PRO R9700
Local AI is setup using distribution mode with nodes.
Describe the bug
When I try and use TTS with a voice that was a referance audio with text using audio-cpp backend with a fish-audio family model I get this error Error: rpc error: code = Internal desc = Fish Audio prepare with inline reference audio requires reference_text option
To Reproduce
My model yaml looks like this.
name: audio-cpp-chatterbox
backend: audio-cpp
flash_attention: true
known_usecases:
- tts
- audio_transform
tts:
voice_cloning: true
parameters:
model: audio-cpp/Fish-Audio-S2-Pro-GGUF/fish-audio-s2-pro-q8_0.gguf
the model is downloaded from https://huggingface.co/audio-cpp/audio.cpp-gguf/tree/main/Fish-Audio-S2-Pro-GGUF
Inside voice library I click "Use in Text to Speech" and then when I try and generate anything it will error Error: rpc error: code = Internal desc = Fish Audio prepare with inline reference audio requires reference_text option
Expected behavior
to be able to use different models with audio-cpp like the fish-audio family of models
Logs
node-backend-logs-audio-cpp-chatterbox-2026-08-20.json
Additional context
There is another PR that is touching ref audio/text inside audio-cpp https://github.com/mudler/LocalAI/pull/11620 if changes are done from master then there might be merge conflicts.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the audio-cpp backend's handling of inline reference audio and reference text, using the provided model YAML and logs to reproduce the Fish Audio error. Review PR #11620 because it touches the same reference-audio/text area; done means the voice-library TTS flow works with Fish Audio models without requiring an unavailable reference_text option.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, go
- Domain
- audio-video-rtc, backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100