Huddles are English-only: neither transcription nor speech can be switched to another language
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
**Type:** feature request (with a concrete blocker)
**Component:** desktop (`huddle/stt.rs`, `huddle/models.rs`, `crates/buzz-voice`)
**Date:** 2026-08-07
**Context:** a German-speaking team running a self-hosted relay, using huddles daily
## Summary
For a team that does not speak English, huddles are currently unusable as a
working tool:
- **Transcription** runs an English-only model. German speech is transcribed as
English phonetics — "Transkribiere bitte deutsche Sprache" comes out as
"Tans three Eera B Deutsche Sach". The huddle transcript is what agents read,
so the agents receive nonsense.
- **Speech** runs an English-only voice model. German replies from agents are
pronounced with English phonetics.
Neither is a setting. Both are pinned in code.
## Transcription
`STT_MODEL_DIR_NAME = "parakeet-tdt-ctc-110m-en"`, artifact
`sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000-int8.tar.bz2`
(`desktop/src-tauri/src/huddle/models.rs:121-127`), pinned by SHA-256. There is
no language selection anywhere in the huddle code — `grep -i "language\|locale"`
over `desktop/src-tauri/src/huddle/` finds nothing relevant.
We tried swapping the model files for the multilingual
`sherpa-onnx-nemo-fast-conformer-ctc-be-de-en-es-fr-hr-it-pl-ru-uk-20k`
(same NeMo-CTC family, so it matches the `nemo_ctc` config in `stt.rs`). The app
accepted it — readiness only checks manifest version and file presence
(`expected_size` is `|_| None` for the STT slot, `models.rs:397,611`) — but the
output was unusable: single Cyrillic characters and periods, no words. Rolled
back. So a drop-in swap is not a viable workaround for operators either.
## Speech
`const MODEL_LANGUAGE: &str = "english_2026-04"`
(`crates/buzz-voice/src/pocket_april.rs:32`), and loading fails outright on a
mismatch:
```
"expected Pocket TTS language {MODEL_LANGUAGE}, got {}"
```
Importing a reference voice (`import_pocket_voice`, 2–30 s of audio) changes the
timbre only — we tested it with a German speaker sample; pronunciation stays
English. There is no alternative TTS backend in the shipped code: a `grep` for
`say|AVSpeech|NSSpeech|espeak|piper` across `desktop/src-tauri/src/huddle/`
returns nothing.
## What would help, in order of usefulness
1. **A `siri:`/system-voice backend.** The key format already appears in tests
(`huddle/tts_settings.rs`: `"siri:aaron"`), and `voice_preferences` is already
a per-backend list. On macOS the system voices cover German (and many other
languages) at good quality, with no extra download. This alone would fix the
speech half for every non-English team.
2. **A language selector for STT**, with per-language model artifacts — even a
small set (de, fr, es) would cover most of Europe.
3. Failing both: **document** that huddles are English-only, so teams do not
spend a day discovering it by experiment.
## Why this matters more than it looks
The huddle transcript is posted into the huddle channel as ordinary `kind:9`
messages, which is what makes agents useful in a voice call — they read what was
said and answer. With an English-only transcriber, a German team's agents receive
garbage, so the whole agent-in-huddle feature collapses for them, not just the
subtitles.
Contributor guide
Research direction
Start by reading desktop/src-tauri/src/huddle/stt.rs and huddle/models.rs to understand the pinned transcription model and model validation, then inspect crates/buzz-voice/src/pocket_april.rs and huddle/tts_settings.rs for the speech backend and existing voice preference format. Done should provide an agreed path for non-English transcription and speech, with language selection or documented limitations covered by appropriate tests or documentation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- macos, rust
- Domain
- audio-video-rtc, desktop
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100