microsoftgraph / microsoftgraph/microsoft-graph-comms-samples
Video Playback through main video or screen share.
Nobody has claimed this yet.
- Dominant language
- No language data
- Stars
- 253
- Forks
- 275
- Avg merge
- 1d 5h
- Merged PRs (30d)
- 1
Description
Describe the issue
I am attempting to record the screen share from a participant then after stopping the recording I want to playback what was recorded through the bot into the meeting.
Code Snippet
In the bot initialization:
`AudioVideoFramePlayerSettings settings = new AudioVideoFramePlayerSettings(new AudioSettings(20), new VideoSettings(), 1000);
framePlayer = new AudioVideoFramePlayer((AudioSocket)this.audioSocket, (VideoSocket)this.mainVideoSocket, settings);`
Pipe buffer from the screen share into a list of buffers:
`List screenShareBufferList = new List();
public void recordScreenShare(VideoMediaBuffer buffer)
{
if (shouldRecordScreenShare)
{
//var newVideo = new VideoSendBuffer(buffer.Data, buffer.Length, VideoFormat.NV12_1920x1080_15Fps, buffer.Timestamp);
screenShareBufferList.Add(buffer);
}
}`
Play the recorded video mostly taken from some other example to try and make it work:
`private const double TicksInOneMs = 10000.0;
private const double MsInOneSec = 1000.0;
public async Task PlayScreenRecord()
{
var videoMediaBuffers = new List();
var referenceTime = DateTime.Now.Ticks;
var packetSizeInMs = (long)((MsInOneSec/ VideoFormat.NV12_1920x1080_15Fps.FrameRate) * TicksInOneMs);
foreach (var buffer in screenShareBufferList)
{
/*nint data = buffer.Data;
IntPtr unManageBuffer = Marshal.AllocHGlobal(data);*/
var videoBuffer = new VideoSendBuffer(buffer.Data, buffer.Length, VideoFormat.NV12_1920x1080_15Fps, referenceTime);
videoMediaBuffers.Add(videoBuffer);
referenceTime += packetSizeInMs;
}
await SendFrameVideo(videoMediaBuffers);
//await SendFrameVideo(screenShareBufferList);
}`
The frame player, I am not sending audio right now, but I feel like this should not be a factor:
`public async Task SendFrameVideo(List buffer)
{
try
{
await framePlayer.EnqueueBuffersAsync(null, buffer);
}
catch (Exception ex)
{
this.logger.Error(ex, $"[OnVideoReceived] Exception while calling SendFrameVideo");
}
}`
Expected behavior
I expect to have video play over the screen share. Currently when I play it will terminate my local debugging session and provides no errors in the try catch block.
Any help would be appreciated, I am not doing this too far away from how the audio is being handled and I can play audio over the meeting. If I attempt to use the vbss socket, I lose all sharing with the bot and still have the same issue of it crashing the app.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the AudioVideoFramePlayer initialization, PlayScreenRecord, and SendFrameVideo code shown in the issue, then compare their VideoSendBuffer and EnqueueBuffersAsync usage with the audio playback example referenced by the reporter. Reproduce the failure while recording and replaying screen-share buffers, and trace the termination outside the reported try/catch. Done means the recorded video plays through the meeting without terminating the debugging session.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- audio-video-rtc
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100