aws / aws/amazon-chime-sdk-cpp
SFU does not announce remote video to peers when local transceiver flips kInactive→kSendRecv via UpdateLocalVideo
- Dominant language
- C++
- Stars
- 23
- Forks
- 7
- PR merge metrics
- No merged PRs in 30d
Description
## Summary
When a peer joins a meeting using the demo's recommended pattern (local video transceiver added with `kInactive`, then later flipped to `kSendRecv`/`kSendOnly` via `SignalingClient::UpdateLocalVideo` + `RtpTransceiverInterface::SetDirectionWithError` + a fresh offer/subscribe cycle), the SFU **intermittently fails to announce that peer's published video as a remote source** to other attendees in the meeting. Other attendees never receive `OnRemoteVideoSourcesAvailable` for the publishing peer, so the video is never displayed.
This affects the C++ SDK's intended use as a bridge/server-side participant. The same call works correctly when the only peers are JS SDK clients.
## Reproduction Context
- C++ Signaling SDK (latest from `main` as of 2026-05-05).
- WebRTC M128 (matches the SDK's pinned version).
- Two peers in the meeting:
1. **C++ SDK bridge** (server-side, our use case): joins with audio + `kInactive` video transceiver. After ICE-Connected, calls `UpdateLocalVideo(mid, {max_bitrate_kbps=1500})` and `SetDirectionWithError(kSendRecv)` followed by a new `CreateOffer` → subscribe cycle. Begins publishing video frames immediately.
2. **JS SDK browser** (Amazon Chime SDK JS, recent version): joins normally with `startLocalVideoTile`.
## Expected Behavior
After the C++ peer's second subscribe cycle (the one that activates video), the JS peer's `audioVideoObserver.videoTileDidUpdate` fires for a remote tile bound to the C++ peer's attendee, and the cart's video stream is rendered.
## Actual Behavior
Intermittently — across many calls in our DEV environment, on the order of 30-50% of the time — the JS peer never receives a video tile for the C++ peer. Sometimes `OnRemoteVideoSourcesAvailable` does fire on the C++ side for the JS peer's video, but the symmetric event never reaches the JS side. The C++ peer is publishing (libwebrtc `outbound-rtp` stats show frames being encoded and sent), but the SFU does not surface it.
When the JS peer manually toggles its local camera off and back on (which forces an SDP renegotiation), the cart's video appears immediately. This is the only reliable workaround we've found.
## Diagnostics
- C++ peer SDP offer (second cycle after activate) includes the video m-line with `a=sendrecv` and the correct payload types.
- The follow-up `Subscribe` reaches the SFU and the `SubscribeAck.sdp_answer` is non-empty (confirming SFU accepted the subscribe).
- libwebrtc `outbound-rtp` stats on the C++ peer show frames being encoded throughout. The SFU is not starving the publisher.
- On the JS side, no `videoTileDidUpdate` ever fires for the C++ peer's attendee until the local camera is toggled.
- After camera toggle, the renegotiation triggers the SFU to announce the C++ peer's video, and from that point on the call is stable.
## Workaround We Landed
Webapp-side: a watchdog that detects "non-self attendee in roster but `framesDecoded` not growing for 6 s" and runs `stopLocalVideoTile()` + `startLocalVideoTile()` to force a renegotiation. This reliably recovers the video.
This is functional but not ideal — a fresh call wastes 6+ seconds of "no cart video" UX before the watchdog fires. We'd much prefer the SFU to announce the remote video source promptly on the C++ peer's second subscribe.
## Possible Root Cause
We suspect the SFU's pairing/announcement logic has a race condition specific to the `kInactive → kSendRecv` flip pattern that the C++ demo recommends (and that is required to avoid the empty-`SubscribeAck.sdp_answer` rejection that occurs when `kSendRecv` is used in the initial offer). When the JS peer is "stable" before the C++ peer's video activation, the SFU sometimes misses announcing the new source.
## Question
1. Is this a known limitation of the C++ SDK's `kInactive → activate` pattern? Is there an alternative pattern that avoids it?
2. Would using `kSendOnly` instead of `kSendRecv` at activation change this behavior? (The demo uses `kSendOnly`; we use `kSendRecv` for unrelated reasons.)
3. Is there a signaling-layer call we should make on the C++ side to explicitly request the SFU re-announce sources to peers, short of forcing a full renegotiation?
Happy to provide more detailed reproduction logs or a minimal repro repo if helpful. Thanks!
Contributor guide
Assessment
This issue has not been assessed yet.