Huddle: receive-side Opus in-band FEC for lossy relay paths
- Dominant language
- Rust
- Stars
- 32.7k
- Forks
- 4.3k
- Avg merge
- 1d 13h
- Merged PRs (30d)
- 253
Description
## Context
Investigated grainy (not laggy) huddle audio over an ordinary residential-ISP path (see internal discussion in #general on the web3services Buzz instance, 2026-08-01). Root cause: the relay's per-peer audio fan-out channel and the client's NetEq jitter buffer both drop/conceal frames under real jitter, and there's currently no recovery mechanism for a genuinely lost frame — only concealment (PLC/expand).
## What's already scoped in-repo
`desktop/src-tauri/src/huddle/jitter.rs:73-76`:
> Receive-side FEC ("decode this frame's redundant copy on a known-lost prior frame") would require either a trait change upstream or NetEq calling decode twice with different `fec` values — out of scope for the initial 10-person fix. Tracked as a follow-up alongside encoder-side `set_inband_fec`.
## Why this needs upstream work, not just a buzz-side change
Confirmed against `neteq` 0.9.0 (latest, we pin 0.8) via docs.rs: `neteq::codec::AudioDecoder::decode(&mut self, encoded: &[u8]) -> Result>` still takes no `fec` parameter and there's no fec-aware decode entry point. Opus in-band FEC only works if the *decoder* is explicitly asked to decode the redundant copy embedded in the *next* packet after a detected loss — the `opus` crate's `Decoder::decode_float` already supports this (it takes a `fec: bool` third argument; buzz's `OpusFrameDecoder::decode` in jitter.rs hardcodes `false`), but nothing in the `neteq` crate's loss-concealment path can currently reach it.
So a real fix needs one of:
1. A fork/patch of the `neteq` crate adding a fec-aware decode path (and using it when NetEq's own loss detection fires), or
2. Upstreaming that capability to `neteq` itself (crates.io owner: darioalessandro).
Either way it also needs encoder-side `opus::Encoder::set_inband_fec(true)` in `desktop/src-tauri/src/huddle/relay_api.rs` — enabling that alone does nothing without the receive-side change, so they must land together.
## Stopgap already shipped
Relay-side per-peer audio channel capacity was widened 8→16 frames (160ms→320ms) in `crates/buzz-relay/src/audio/room.rs` to reduce hard `try_send` drops under jitter close to the old ceiling. This does not recover lost audio content — it just reduces how often frames are dropped outright. See PR (link once open).
Contributor guide
Research direction
Start with desktop/src-tauri/src/huddle/jitter.rs:73-76 and inspect neteq 0.8's AudioDecoder API and loss-concealment path. Then review desktop/src-tauri/src/huddle/relay_api.rs for encoder configuration and determine whether an upstream neteq change or a fork is needed. Done means receive-side Opus FEC and encoder-side in-band FEC work together; the relay channel widening is already shipped.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100