alphacep / alphacep/vosk-api

iOS: Audio decoding restricted to microphone input source only

Open
#2,017 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
15.1k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

An iOS app uses a WKWebView inside a UIViewController to run a web-based game within my app (Swift framework).

**Requirements**
• Vosk must recognise only the input source as mic only.
• Vosk must not detect or transcribe any audio coming from the game inside the WKWebView. **(WKWebView used in my app. Not other app)**

For example, Vosk should not transcribe the WKWebView audio (game sounds, music, voice, etc.). It should only process the user’s spoken voice from the mic input and send that mic audio to the vosk speech decoder.

**Question**

How can I modify my iOS Swift code to make Vosk listen only to the microphone and ignore the WKWebView/game audio?

Note: This is required for an iOS app.

**Code:**

```
// Create the audio engine
self.audioEngine = AVAudioEngine()

// Get the microphone input node + its format
let inputNode = audioEngine?.inputNode
let inputFormat = inputNode?.inputFormat(forBus: 0)
let pcmFormat = AVAudioFormat.init(
commonFormat: AVAudioCommonFormat.pcmFormatInt16,
sampleRate: inputFormat?.sampleRate ?? 0.0,
channels: 1,
interleaved: true
)

// Load/create the Vosk model
let model = VoskModel()
guard var words = words?.map({ $0.lowercased() }), !words.isEmpty else {
throw RecognizerError.noAnswersProvided
}

// Vosk recognizer
guard let recognizer = Vosk(model: model, sampleRate: Float(inputFormat?.sampleRate ?? 0.0), wordsJSON: wordsJSON) else {
throw RecognizerError.recognizerIsUnavailable
}

// Install a “tap” on the mic input node
inputNode?.installTap(
onBus: 0,
bufferSize: UInt32((inputFormat?.sampleRate ?? 0.0) / 20),
format: pcmFormat
) { [weak self] buffer, time in
// result
})
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.