block / block/buzz

Huddle transcription is English-only: non-English speech (e.g. Korean) produces garbled transcripts

Open
#2,478 4 comments 1 reaction 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

## Summary

Huddle transcription produces garbled output for any non-English speech. The embedded STT model is hardcoded to an English-only model, with no language setting or override.

## What happens

Speaking Korean in a huddle yields transcripts like:

> "You know b H texture Isam an onoroso Chen."
> "I to go to Tantaa"

The model force-decodes Korean audio into English tokens, so the output is unusable. English speech transcribes fine.

## Root cause

The STT model is pinned to NVIDIA Parakeet TDT-CTC 110M **English-only** in `desktop/src-tauri/src/huddle/models.rs`:

- `STT_MODEL_DIR_NAME: &str = "parakeet-tdt-ctc-110m-en"` (models.rs:131)
- Download archive pinned to `sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000-int8.tar.bz2` with a checksum (models.rs:125-128)

There is no language option, env override, or model-swap path — the loader only knows this one model.

## Proposal

The embedded engine (sherpa-onnx) already supports multilingual models, so this is achievable without changing the STT stack. For example, **SenseVoice-Small** handles Korean, English, Japanese, Chinese, and Cantonese at a comparable size and is available in sherpa-onnx's model catalog.

Options, roughly in order of effort:

1. **Add a transcription-language setting** that selects between the current English model and a multilingual model (downloaded on demand, same flow as today).
2. **Auto-select by system locale** — non-English locales get the multilingual model by default.
3. Minimal: allow an advanced/config override of the model dir + type so users can supply their own sherpa-onnx model.

Happy to help test Korean transcription on a build with a multilingual model.

## Environment

- Buzz Desktop on macOS (Darwin 25.5.0, Apple Silicon)
- Speaker language: Korean

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.