microsoftgraph / microsoftgraph/microsoft-graph-comms-samples

Video Playback through main video or screen share.

Open
#762 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
253
Forks
275
Avg merge
1d 5h
Merged PRs (30d)
1

Description

Describe the issue
I am attempting to record the screen share from a participant then after stopping the recording I want to playback what was recorded through the bot into the meeting.

Code Snippet
In the bot initialization:
`AudioVideoFramePlayerSettings settings = new AudioVideoFramePlayerSettings(new AudioSettings(20), new VideoSettings(), 1000);

framePlayer = new AudioVideoFramePlayer((AudioSocket)this.audioSocket, (VideoSocket)this.mainVideoSocket, settings);`

Pipe buffer from the screen share into a list of buffers:
`List screenShareBufferList = new List();

public void recordScreenShare(VideoMediaBuffer buffer)
{
if (shouldRecordScreenShare)
{
//var newVideo = new VideoSendBuffer(buffer.Data, buffer.Length, VideoFormat.NV12_1920x1080_15Fps, buffer.Timestamp);
screenShareBufferList.Add(buffer);
}
}`

Play the recorded video mostly taken from some other example to try and make it work:
`private const double TicksInOneMs = 10000.0;
private const double MsInOneSec = 1000.0;

public async Task PlayScreenRecord()
{
var videoMediaBuffers = new List();
var referenceTime = DateTime.Now.Ticks;

var packetSizeInMs = (long)((MsInOneSec/  VideoFormat.NV12_1920x1080_15Fps.FrameRate) * TicksInOneMs);

foreach (var buffer in screenShareBufferList)
{

    /*nint data = buffer.Data;
    IntPtr unManageBuffer = Marshal.AllocHGlobal(data);*/
    var videoBuffer = new VideoSendBuffer(buffer.Data, buffer.Length, VideoFormat.NV12_1920x1080_15Fps, referenceTime);
    videoMediaBuffers.Add(videoBuffer);
    referenceTime += packetSizeInMs;
}
await SendFrameVideo(videoMediaBuffers);
//await SendFrameVideo(screenShareBufferList);

}`

The frame player, I am not sending audio right now, but I feel like this should not be a factor:
`public async Task SendFrameVideo(List buffer)
{
try
{

    await framePlayer.EnqueueBuffersAsync(null, buffer);
}
catch (Exception ex)
{
    this.logger.Error(ex, $"[OnVideoReceived] Exception while calling SendFrameVideo");
}

}`

Expected behavior
I expect to have video play over the screen share. Currently when I play it will terminate my local debugging session and provides no errors in the try catch block.

Any help would be appreciated, I am not doing this too far away from how the audio is being handled and I can play audio over the meeting. If I attempt to use the vbss socket, I lose all sharing with the bot and still have the same issue of it crashing the app.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the AudioVideoFramePlayer initialization, PlayScreenRecord, and SendFrameVideo code shown in the issue, then compare their VideoSendBuffer and EnqueueBuffersAsync usage with the audio playback example referenced by the reporter. Reproduce the failure while recording and replaying screen-share buffers, and trace the termination outside the reported try/catch. Done means the recorded video plays through the meeting without terminating the debugging session.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
audio-video-rtc
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.