jamulussoftware / jamulussoftware/jamulus
Raw audio has no packet-loss concealment
- Dominant language
- C
- Stars
- 1.1k
- Forks
- 248
- Avg merge
- 2d 3h
- Merged PRs (30d)
- 9
Description
**🤖 AI:** A lost raw-audio packet and a lost Opus packet are handled differently a few lines apart in [the same loop](https://github.com/jamulussoftware/jamulus/blob/b3c93588bb447d2c47987aba35284131a3e3c456/src/server.cpp#L981-L1001): a missing Opus frame goes to [`opus_custom_decode`](https://github.com/jamulussoftware/jamulus/blob/b3c93588bb447d2c47987aba35284131a3e3c456/src/server.cpp#L986-L990) with a null pointer, which is packet loss concealment, while a missing raw frame is [`memset`](https://github.com/jamulussoftware/jamulus/blob/b3c93588bb447d2c47987aba35284131a3e3c456/src/server.cpp#L1001) to zero — a hard digital gap. The consequence is measurable, audible, and present with no loss injected at all.
Measured with one client sending a 440 Hz sine, once as raw PCM and once as Opus, loss injected at the sender, the server's own `-R` recording as the instrument. Silence gaps of 64 samples or more, counted over 17 s of steady state:
| injected loss | raw: gaps/s | raw: zero samples | Opus: gaps/s | Opus: zero samples |
|---:|---:|---:|---:|---:|
| 0% | 2.1 | 0.81% | 0.0 | 0.01% |
| 2% | 9.2 | 3.20% | 0.0 | 0.01% |
| 5% | 16.8 | 4.85% | 0.0 | 0.01% |
At 2% loss the longest single raw gap ran 1664 samples, 35 ms.
Repeating it with a synthesised musical phrase, and with the Opus arm at `AQ_HIGH`'s real 82 coded bytes rather than a size picked to avoid a collision, gives the same split for raw: 0.6 discontinuities/s at 0% injected loss, 7.9/s at 2%, 18.5/s at 5%, with the jump to zero reaching 3.0–3.8× the signal RMS. A listener judging the recordings by ear counted 8 audible clicks in the 0%-loss raw recording (the detector found 9), "many" at 2%, "very many" at 5%.
### Both modes on one scale
Counting discontinuities favours raw's failure mode: a zero-fill leaves a sample-level jump, concealment leaves a smooth but wrong signal, and the same listener reported weak artifacts in the `AQ_HIGH` recordings that no discontinuity detector scored above zero. Comparing against the signal that was sent measures both. Per 20 ms frame, log-spectral distance across 24 bands, aligned on chirp markers embedded once per second, a frame counted bad at 8 dB. Two runs of each arm:
| injected loss | raw: bad frames | `AQ_HIGH`: bad frames |
|---:|---:|---:|
| 0% | 4.5%, 3.6% | 1.4%, 1.1% |
| 2% | 12.0%, 8.3% | 5.3%, 5.0% |
| 5% | 21.3%, 19.6% | 12.3%, 13.2% |
Raw is worse at every rate in both runs, including with nothing injected. `AQ_HIGH` is not clean either — 1.4% and 1.1% of frames are wrong with no loss injected, which is what the listener described for those same files. The difference is in kind as well as degree, and only the discontinuity is what a listener called jarring.
### The same link loses more raw packets in the first place
The two modes send at the *same packet rate* — one packet per 2.67 ms, ~375/s. Only the size differs: 257 bytes for raw mono against 83 for `AQ_HIGH` mono. So this is not "more packets to lose", it is "more bytes to fit", and it only bites below a threshold. Measured on a shaped link (`tbf`, client→server, server in its own network namespace), counting audio datagrams that arrived:
| shaped link | raw, 257 B/pkt | `AQ_HIGH`, 83 B/pkt |
|---|---:|---:|
| 400 kbit | 52.0% loss | 0.0% |
| 600 kbit | 28.7% loss | 0.0% |
| 900 kbit | 0.0% | 0.0% |
| 1500 kbit | 0.0% | 0.0% |
At 400 and 600 kbit the choice of quality setting decides whether audio arrives at all; by 900 kbit both fit. A repeat of the two shaped points returned 51.1% and 27.5%, so the run-to-run spread is about one point. That is the range where the two effects compound: raw loses the packets *and* has no concealment for the ones it loses, while `AQ_HIGH` on the same link loses none.
Concealment for the raw path is the obvious question, and it is not free — the decoder has no model of the signal to conceal with, which is exactly what Opus provides. Repeating the previous frame, or a short fade to silence instead of an instant `memset`, would remove the discontinuity without one.
Worth noting for anyone reproducing this: a discontinuity detector cannot compare the two modes, because it can only see one of them. Three detectors here scored the `AQ_HIGH` recordings at zero while a listener heard artifacts in them, and the reference comparison above is what resolved the disagreement.
Rig, harness and the full method are available if useful. How a server tells raw from Opus in the first place is a separate matter, covered in [#3894](https://github.com/jamulussoftware/jamulus/issues/3894).
---
🤖 *This message was written by AI and reviewed by @mcfnord.*
Contributor guide
Research direction
Start in src/server.cpp around lines 981-1001 and compare the missing raw-frame memset path with the missing Opus-frame decode path. Review the issue's proposed repeat-frame or fade-to-silence options and the measurements before choosing behavior; done means raw packet loss no longer creates an immediate hard zero-filled gap without regressing the existing audio paths.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- c
- Domain
- audio-video-rtc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 48/100