The joining backfill costs a base-layer GOP, on exactly the links that cannot spare it
- Dominant language
- Go
- Stars
- 1
- Forks
- 1
- Avg merge
- 7m
- Merged PRs (30d)
- 4
Description
Deferred from #4. Not urgent — the links this is running on today are fine — but
the shape of the problem is worth writing down while it is fresh, because the
fix is a design choice rather than a repair, and the choice is coupled to the
object-ID inversion in a way that is not obvious later.
## What it costs
`backfillGroup` (`internal/conf/remote.go`) replays the group in progress with a
Joining FETCH, narrowed to the base layer by `SUBGROUP_FILTER`. A subscriber
joining at a random point in a five-second GOP pulls, on average, ~2.5 s of
base-layer video — order of a couple of hundred kilobytes at the 1.5 Mbps the
ladder targets.
That is cheap on a healthy link and it is the whole reason a joining tile paints
at all. On a constrained one it is neither.
## Why the constrained case is worse than it first looks
**It fires on every resubscribe, not just on join.** Layer change, demotion,
tile scrolled back into view — and, most importantly, the lag resync: past
`maxLag` (1.5 s) the subscription is rebuilt at the live edge, rate limited by
`resyncCooldown` to once every 15 s. So a link that is already failing to hold
the live edge answers that by spending a fresh base-layer GOP, every fifteen
seconds, on top of the live media it was already failing to carry. The response
to "we are behind" is more bytes.
**On a slow enough link it pays the cost and gets nothing.**
`videoBackfillTimeout` bounds how long live video waits behind the backfill at
3 s. A base-layer GOP does not cross a 32 kB/s link in three seconds — the
bottleneck the `degraded_test.go` shapers model — so the wait expires, live is
released, and the bytes already spent were spent for a picture that never
arrived. Worst of both.
**Priority cannot arbitrate it.** The obvious answer is to rank the backfill
below live audio and video with `SUBSCRIBER_PRIORITY`, and it would be correct
protocol behaviour — but see the priority constants at the top of `remote.go`:
moq-go's adapters implement no per-stream scheduling knob, because quic-go
exposes none, so §10.2.7 changes nothing measurable on either transport this app
can use. Stamping a lower priority would be honest and inert.
## The fork
The two ends of it are not a matter of tuning; they want different transports.
**Full-GOP backfill (what ships).** The subscriber gets the whole reference
chain, so the live continuation of that group is decodable and the picture runs
continuously from the moment it paints. This is what the object-ID inversion in
#4 buys — base layer on the top range so a largest-object subscription keeps
receiving it mid-group.
**Keyframe-only backfill.** Fetch object zero of the base range and nothing
else: one object, near enough free. The tile paints a still immediately and
motion resumes at the next group, which `requestNewGroup` has already asked for,
so roughly one RTT away on a healthy link. But the frames between the keyframe
and the live edge are then missing, so the live continuation references frames
the decoder never got — which means it must be discarded, **and then the
object-ID inversion buys nothing at all**. Reverting it would be a second wire
break.
That coupling is the thing to remember: picking keyframe-only later is not a
narrowing of the current design, it is undoing part of it.
## What would decide it
Nothing here should be chosen from reasoning; it wants numbers.
- What a joining backfill actually costs in bytes and in wall-clock, measured
per participant on a real call rather than derived from the bitrate target.
- Whether the resync loop above is observable — does a struggling link visibly
spend its recovery on backfills? The telemetry counters already separate
inbound tracks, so a backfill's bytes should be attributable.
- Whether the 3 s timeout is ever *reached* in practice, which is the sharpest
single signal: a backfill that times out is pure waste, and if it never times
out the constrained case is theoretical.
## Middle options, if the numbers say so
- Skip the backfill and lean on `NEW_GROUP_REQUEST` alone when the drift meter
says the path is already behind — the information is local and already
measured (`checkLag`).
- Bound the backfill by age rather than by group: fetch the tail of the base
layer plus the keyframe. Needs the reference chain to be complete, so it only
works if the encoder's GOP has no long-term references beyond the keyframe,
which for the current `L1T2`/H.264 configuration it does not — worth checking
rather than assuming.
- Keep the full backfill but make the timeout adaptive, so a link that cannot
finish one stops starting them.
## Not blocking
Filed as a known trade rather than a defect. The behaviour on a healthy link is
the intended one, and the constrained path degrades to what 0.7.2 did — a blank
tile until the next keyframe — rather than to anything worse.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start in internal/conf/remote.go at backfillGroup, SUBGROUP_FILTER, the priority constants, videoBackfillTimeout, resyncCooldown, and checkLag. Run or inspect degraded_test.go and the existing inbound-track telemetry, then measure backfill bytes, wall-clock time, resync behavior, and timeout frequency per participant. Done means the measurements support a documented transport or backfill decision.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- audio-video-rtc
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Quiet
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100