ROCm / ROCm/FastFlowLM

Bad whisper speech recognition ( german )

Open
#313 2 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
1.9k
Forks
152
Avg merge
4h 14m
Merged PRs (30d)
11

Description

Hi guys,
found your project and find it absolute interesting.
Running an ryzen ai max+ 395 for my local ki tasks and trying to go completly local for my home assistant voice assistants.
For now i am using ollama with qwen3-vl:30b-a3b-instruct as agent, piper for tts and home assistant nabu casa for stt ( as i know they are also using whisper ). This combination runs absolut perfect.

The last piece for going complete local is a local whisper instance for stt so i gave your solution a try.

It works, but the speech recognition is not very good. Trying to "schalte das Licht in der Küche aus" ends up with "Scheiße das Licht in der Küche aus". It understands "scheiße" all the time wen i said "schalte" That is very bad for a smart home in germany :-)

Starting server with "C:\Windows\System32\cmd.exe /K "C:\Program Files\flm\flm.exe" serve -a 1 --host 0.0.0.0"
HA connects in open ai mode.

configuration.yaml:
stt:

  • platform: openai_stt
    api_key: 1234
    realtime: false
    api_url: http://192.168.178.182:52625/v1
    #model: whisper-v3
    prompt: ""
    temperature: 0
    noise_reduction: null

Is this a problem with the whisper npu model?

Image

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the German phrase using the server command in the issue and the configuration.yaml OpenAI STT settings. Compare the resulting transcription with the reported “schalte”/“Scheiße” substitution and determine whether the behavior is specific to the Whisper NPU model; done means the model cause is established or ruled out with supporting output.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, yaml
Domain
ai, audio-video-rtc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.