microsoftgraph / microsoftgraph/microsoft-graph-comms-samples

Transcribe 'UnmixedMeetingAudio' buffer as soon as those are received.

Open
#430 10 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
No language data
Stars
253
Forks
275
Avg merge
1d 5h
Merged PRs (30d)
1

Description

This is a question to the community but not an issue. I am able to receive unmixed audio in AudioMediaReceived event handler inside CallHandler.cs class, during P2P or group call/online meeting. Now I want to generate transcript out of it. I want some guidance about:

  1. Can this be done at the end of the meeting, I am confused as it's stated on a number of places that

"You may not use this SDK to record or otherwise persist media content from calls or meetings that your bot accesses" .

In which scenario this fits? Is to possible to generate wave file out of the byte array (UnmixedAudioBuffer data) received and use that for transcription at the end of the meeting?

  1. Since we receive audio frames at a very quick pace (50 frames per second), If I am to generate the transcript asynchronously and real time, how should I do it. Is it about converting buffer data to .wav or in memory stream and using some cloud speech service/API?
    There's a speech service provided by Microsoft as well as speech to text resource provided by Google cloud.
    your guidance is highly appreciated.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the AudioMediaReceived event handler in CallHandler.cs and the UnmixedAudioBuffer data described in the issue. Review the SDK restriction on recording or persisting call media before assessing the proposed end-of-meeting and real-time transcription approaches. Done would be clear guidance on whether either workflow is permitted and how it should handle the received audio frames.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
audio-video-rtc, cloud
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.