alphacep / alphacep/vosk-api

iOS: Audio decoding restricted to microphone input source only

Abierto
#2,017 0 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Jupyter Notebook
Estrellas
15.1k
Forks
1.8k
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

An iOS app uses a WKWebView inside a UIViewController to run a web-based game within my app (Swift framework).

**Requirements**
• Vosk must recognise only the input source as mic only.
• Vosk must not detect or transcribe any audio coming from the game inside the WKWebView. **(WKWebView used in my app. Not other app)**

For example, Vosk should not transcribe the WKWebView audio (game sounds, music, voice, etc.). It should only process the user’s spoken voice from the mic input and send that mic audio to the vosk speech decoder.

**Question**

How can I modify my iOS Swift code to make Vosk listen only to the microphone and ignore the WKWebView/game audio?

Note: This is required for an iOS app.

**Code:**

```
// Create the audio engine
self.audioEngine = AVAudioEngine()

// Get the microphone input node + its format
let inputNode = audioEngine?.inputNode
let inputFormat = inputNode?.inputFormat(forBus: 0)
let pcmFormat = AVAudioFormat.init(
commonFormat: AVAudioCommonFormat.pcmFormatInt16,
sampleRate: inputFormat?.sampleRate ?? 0.0,
channels: 1,
interleaved: true
)

// Load/create the Vosk model
let model = VoskModel()
guard var words = words?.map({ $0.lowercased() }), !words.isEmpty else {
throw RecognizerError.noAnswersProvided
}

// Vosk recognizer
guard let recognizer = Vosk(model: model, sampleRate: Float(inputFormat?.sampleRate ?? 0.0), wordsJSON: wordsJSON) else {
throw RecognizerError.recognizerIsUnavailable
}

// Install a “tap” on the mic input node
inputNode?.installTap(
onBus: 0,
bufferSize: UInt32((inputFormat?.sampleRate ?? 0.0) / 20),
format: pcmFormat
) { [weak self] buffer, time in
// result
})
```

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.