block / block/buzz

Huddles are English-only: neither transcription nor speech can be switched to another language

Open
#5,194 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
32.7k
Forks
4.3k
Avg merge
1d 13h
Merged PRs (30d)
253

Description

**Type:** feature request (with a concrete blocker)
**Component:** desktop (`huddle/stt.rs`, `huddle/models.rs`, `crates/buzz-voice`)
**Date:** 2026-08-07
**Context:** a German-speaking team running a self-hosted relay, using huddles daily

## Summary

For a team that does not speak English, huddles are currently unusable as a
working tool:

- **Transcription** runs an English-only model. German speech is transcribed as
English phonetics — "Transkribiere bitte deutsche Sprache" comes out as
"Tans three Eera B Deutsche Sach". The huddle transcript is what agents read,
so the agents receive nonsense.
- **Speech** runs an English-only voice model. German replies from agents are
pronounced with English phonetics.

Neither is a setting. Both are pinned in code.

## Transcription

`STT_MODEL_DIR_NAME = "parakeet-tdt-ctc-110m-en"`, artifact
`sherpa-onnx-nemo-parakeet_tdt_ctc_110m-en-36000-int8.tar.bz2`
(`desktop/src-tauri/src/huddle/models.rs:121-127`), pinned by SHA-256. There is
no language selection anywhere in the huddle code — `grep -i "language\|locale"`
over `desktop/src-tauri/src/huddle/` finds nothing relevant.

We tried swapping the model files for the multilingual
`sherpa-onnx-nemo-fast-conformer-ctc-be-de-en-es-fr-hr-it-pl-ru-uk-20k`
(same NeMo-CTC family, so it matches the `nemo_ctc` config in `stt.rs`). The app
accepted it — readiness only checks manifest version and file presence
(`expected_size` is `|_| None` for the STT slot, `models.rs:397,611`) — but the
output was unusable: single Cyrillic characters and periods, no words. Rolled
back. So a drop-in swap is not a viable workaround for operators either.

## Speech

`const MODEL_LANGUAGE: &str = "english_2026-04"`
(`crates/buzz-voice/src/pocket_april.rs:32`), and loading fails outright on a
mismatch:

```
"expected Pocket TTS language {MODEL_LANGUAGE}, got {}"
```

Importing a reference voice (`import_pocket_voice`, 2–30 s of audio) changes the
timbre only — we tested it with a German speaker sample; pronunciation stays
English. There is no alternative TTS backend in the shipped code: a `grep` for
`say|AVSpeech|NSSpeech|espeak|piper` across `desktop/src-tauri/src/huddle/`
returns nothing.

## What would help, in order of usefulness

1. **A `siri:`/system-voice backend.** The key format already appears in tests
(`huddle/tts_settings.rs`: `"siri:aaron"`), and `voice_preferences` is already
a per-backend list. On macOS the system voices cover German (and many other
languages) at good quality, with no extra download. This alone would fix the
speech half for every non-English team.
2. **A language selector for STT**, with per-language model artifacts — even a
small set (de, fr, es) would cover most of Europe.
3. Failing both: **document** that huddles are English-only, so teams do not
spend a day discovering it by experiment.

## Why this matters more than it looks

The huddle transcript is posted into the huddle channel as ordinary `kind:9`
messages, which is what makes agents useful in a voice call — they read what was
said and answer. With an English-only transcriber, a German team's agents receive
garbage, so the whole agent-in-huddle feature collapses for them, not just the
subtitles.

Contributor guide

Open the contributing guide

Research direction

Start by reading desktop/src-tauri/src/huddle/stt.rs and huddle/models.rs to understand the pinned transcription model and model validation, then inspect crates/buzz-voice/src/pocket_april.rs and huddle/tts_settings.rs for the speech backend and existing voice preference format. Done should provide an agreed path for non-English transcription and speech, with language selection or documented limitations covered by appropriate tests or documentation.

Written by the indexing model from the issue text.

Assessment

Tech stack
macos, rust
Domain
audio-video-rtc, desktop
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.