BabylonJS / BabylonJS/BabylonNative
iOS/JSC: getUserMedia camera VideoTexture never renders — NativeVideo::SetSrcObject rejects MediaStream (InstanceOf fails across napi/JSC)
- Dominant language
- C++
- Stars
- 919
- Forks
- 162
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 19
Description
### Summary
On iOS (JavaScriptCore), `NativeVideo::SetSrcObject` rejects the `MediaStream` returned by `navigator.mediaDevices.getUserMedia`, so `VideoTexture.CreateFromStreamAsync` never renders the camera feed. Camera *capture* works (AVCaptureSession runs, frames flow), but no texture ever updates.
### Environment
- Babylon Native `master` @ `64f545de`; Babylon.js `9.9.1` (UMD `babylon.max.js`)
- iOS 26.5, iPhone 15 Pro Max (device), Xcode 26
- Repro: `Apps/Playground` iOS with the `cameraTexture` path in `experience.js` enabled
### Root cause
`Plugins/NativeCamera/Source/NativeVideo.cpp`, `SetSrcObject`:
```cpp
if (value.IsNull() || value.IsUndefined() ||
!value.As().InstanceOf(MediaStream::GetConstructor(env)))
{
// reject: m_streamObject cleared, m_isReady = false
return;
}
```
`getUserMedia` creates the stream via `MediaStream::GetConstructor(env).New({})`, and `SetSrcObject` checks `InstanceOf(MediaStream::GetConstructor(env))` against the same cached constructor — yet **`InstanceOf` returns false** across the napi/JavaScriptCore boundary for the genuine `MediaStream` object.
On-device trace (instrumented):
```
SetSrcObject called null=0 obj=1 instanceOfMediaStream=0 <- object is a MediaStream, InstanceOf fails
```
Because the stream is rejected, `readyState` stays `0`, so Babylon.js's `VideoTexture.CreateFromStreamAsync` never gets the `"playing"` event (its `NativeVideo::Play` only fires `"playing"` when a stream is set), the returned Promise never resolves, the `VideoTexture` is never constructed, and `engine.updateVideoTexture` is never called.
### Impact
`getUserMedia` + `VideoTexture.CreateFromStreamAsync` (camera-to-texture) is completely broken on the JSC/iOS backend.
### Suggested fix
Either investigate why `Napi::Value::InstanceOf` fails for ObjectWrap constructors on the JSC engine, or identify the `MediaStream` more robustly. A pragmatic fix is to duck-type on the presence of `getVideoTracks`:
```cpp
const bool looksLikeMediaStream =
value.IsObject() && value.As().Get("getVideoTracks").IsFunction();
if (value.IsNull() || value.IsUndefined() || !looksLikeMediaStream) { /* reject */ }
```
Happy to open a PR.
Contributor guide
Research direction
Start with Apps/Playground on iOS using the cameraTexture path in experience.js, then inspect NativeVideo::SetSrcObject in Plugins/NativeCamera/Source/NativeVideo.cpp and trace the failing InstanceOf check on JavaScriptCore. Done means the genuine getUserMedia MediaStream is accepted, the playing event occurs, CreateFromStreamAsync resolves, and the camera VideoTexture renders on iOS.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, ios, javascript
- Domain
- computer-graphics, mobile-dev
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 48/100