Unity-Technologies / Unity-Technologies/com.unity.webrtc

[REQUEST]: Timestamp for sent/received video frames (pts/dts)

Open
#578 4 comments 0 reactions 1 assignee View on GitHub

@karasusan is already working on this.

Since Dec 28, 2021.

enhancement issued
Dominant language
Assembly
Stars
852
Forks
238
PR merge metrics
No merged PRs in 30d

Description

Is your feature request related to a problem?

It would be nice to have some way to synchronize video frames with some other data, e.g. subtitles, transforms, etc. Currently there is no way to figure out what is the frame I'm receiving, they only come in sequence and I can't tell when exactly this frame was rendered.

Describe the solution you'd like

Can we have a callback when the frame goes to ecoder, with its uniques pts or dts? Can we access pts or dts on the receiving side?

Describe alternatives you've considered

I tried to rely on the sequence of frames I'm sending/receiving, but you still can't say if the frame was dropped, and you can't tell when exactly the render texture was accessed and fed to the encoder, because encoder and Unity run at different framerates.

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.