Unity-Technologies / Unity-Technologies/com.unity.webrtc

[BUG]: No data sent on VideoStreamTracks on Pico after multiple cleanups

Open
#1,020 2 comments 1 reaction 1 assignee View on GitHub

@karasusan is already working on this.

Since Mar 5, 2024.

bug issued
Dominant language
Assembly
Stars
852
Forks
238
PR merge metrics
No merged PRs in 30d

Description

Package version

3.0.0-pre.7

Environment
* OS: Android
* Unity version: 2020.3
Steps To Reproduce
  1. Create a VideoStreamTrack and attach to a MediaStream
  2. Attach stream to 30 connections
  3. Teardown connections and tracks/channels
  4. Create another VideoStreamTrack and attach to a MediaStream
  5. Attach stream to any number of connections

One way we've reproduced this outside of our app: Build Samples/WebRTC/3.0.0-pre.7/Example/VideoReceive for pico, call, add track, remove track, add track, remove track, add track, remove track, etc (because there's a single texture, you'll need to do it about 30 times. I needed to do it 32 times), hang up and then call again. You'll see the receiving texture turn black.

Current Behavior

No data is sent on the recreated stream.

Expected Behavior

The new media streams work like the old ones and stream images.

Anything else?

We have an application that runs on a Pico Neo 3 Pro Eye. Our web portal establishes a connection with our HMD, and our HMD will create two MediaStreams, each with a VideoStreamTrack attached (one for each of the inward-facing camera feeds). This seems to work beautifully. We can support 50+ connections (all sharing the same two MediaStreams), no problem. They can disconnect, and new connections can form.

We can successfully stop and teardown our MediaStreams and connections and then create new connections with new MediaStreams. Everything is hunky dory until we've had about 16-20 connections (over 1-20 separate pairs of MediaStreams). The next time we try to create new MediaStreams and stream our videos with any number of connections, even though we appear to successfully add our tracks to our connection(s), we don't send any image data.

  • Our teardown process is by-the-book:
  1. Stop WebRTC.Update loop
  2. Stop all tracks
  3. Close data channels in all connections
  4. Remove tracks from all connections
  5. Close all connections
  6. Wait for connections to close
  7. Remove callbacks from Connection
  8. Remove data channel callbacks
  9. Dispose of channels in all connections
  10. Dispose of senders in all connections
  11. Dispose of transceivers in all connections
  12. Remove callbacks from all connections
  13. Dispose of all peer connections
  14. Remove callbacks from Signaling Servers
  • Even when the videos aren't sending, we're able to send data through data channels (we successfully renegotiate upon adding our tracks)
  • The VideoStreamTracks exist on both sides of the connection
  • We get no exceptions unless we force our app to use a codec other than VP8, in which case we get similar behavior but also a seg fault in the WebRTC.Update() method in batch.Submit()
  • Our targetBitrate for failed video streams is 144880, whereas it's 600000 for successful streams
  • Sometimes only one of the two MediaStreams will fail
  • The textures are 200x200
  • Once we have a failure, we need to restart our application before we can successfully send tracks again

We suspect something related to closing Peer Connections or releasing graphics memory. Please let me know what else I can give you. Any guidance will be greatly appreciated!

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.