How to use Vosk in C#
- Lenguaje dominante
- Jupyter Notebook
- Estrellas
- 15.1k
- Forks
- 1.8k
- Métricas de merge de PR
- Sin PR fusionados en 30 d
Descripción
So far, I tried 3 different approaches to achieve dictation in a way Microsoft does it in Word with Dictate button.
I used the following:
1. .NET Framework classes such as SpeechRecognitionEngine from System.Speech.Recognition
2. I used Vosk with output to the screen
3. I used Vosk with output to audio file (*.wav)
All the examples below were recognized perfectly by Word Dictation.
#1 was pretty bad - it could recognized just a few words and the rest was nonsense.
#2 I had great hopes for it but it was bad as well, here are some results, and believe me I had great headset mic and I spoke very clearly and a bit slower to separate between words...
I said Output
"This is the first example" "the for example"
"This is the second example" "the descending bull"
"I do not understand what is going on" "i don't understand what going on"
“So, what is I am doing wrong here" "the what is in the winter"
"Hello there, are you alive?" "the previous peak of the blood"
"Why the speech recognition is so bad?" "the white asparagus is bad"
As you can see, it is a complete nonsense as well.
Therefore, I am asking myself: "What did I do wrong?"
I downloaded and pointing to the model folder of "vosk-model-en-us-0.22" (BTW, initialization takes 15 seconds!)
Here is the code, it followed the example I got here that used PortAudioSharp library from NuGet.
Based on this I did not even try generating audio file using NAudio...
Can anybody tell me what am I doing wrong? (I have a slight German accent in English) I assume here that all the projects that are using vosk has better quality...
How MS Word doing it with Dictate? What are they using?
When I click on the button in my WinForms app, the following is executed:
```
string voskModelFolder = Path.Combine(Program.InstallDir, @"Gpt4AudioData\Models\vosk-model-en-us-0.22");
VoskPortAudioSharp voskPortAudio = new VoskPortAudioSharp();
voskPortAudio.SpeechRecognizedEvent += VoskPortAudio_SpeechRecognizedEvent;
voskPortAudio.Init(voskModelFolder);
voskPortAudio.Start();
```
and the functions:
```
public void Init(string modelFolder)
{
using (Log.VerboseCall())
{
try
{
Model model = new Model(modelFolder);
recognizer = new VoskRecognizer(model, 16000.0f);
PortAudio.LoadNativeLibrary();
PortAudio.Initialize();
oParams.device = PortAudio.DefaultInputDevice;
if (oParams.device == PortAudio.NoDevice)
{
string err = "No default audio input device available";
Log.Verbose(err);
throw new Exception(err);
}
oParams.channelCount = 1;
oParams.sampleFormat = SampleFormat.Int16;
oParams.hostApiSpecificStreamInfo = IntPtr.Zero;
var callbackData = new VoskCallbackData()
{
textResult = String.Empty
};
mStream = new PortAudioSharp.Stream(
oParams,
null,
16000,
8192,
StreamFlags.ClipOff,
playCallback,
callbackData
);
}
catch (Exception ex)
{
Log.Verbose(ex);
}
}
}`
` public void Start()
{
using(Log.VerboseCall())
{
try
{
mStream.Start();
Log.Verbose("Started...");
}
catch(Exception ex)
{
Log.Verbose(ex);
}
}
}
public void Stop()
{
using(Log.VerboseCall())
{
try
{
mStream.Stop();
Log.Verbose("Stopped");
}
catch(Exception ex)
{
Log.Verbose(ex);
}
}
}
class VoskCallbackData
{
public String textResult { get; set; }
}
private StreamCallbackResult playCallback( IntPtr input,
IntPtr output,
System.UInt32 frameCount,
ref StreamCallbackTimeInfo timeInfo,
StreamCallbackFlags statusFlags,
IntPtr dataPtr )
{
try
{
byte[] buffer = new byte[frameCount];
Marshal.Copy(input, buffer, 0, buffer.Length);
System.IO.Stream streamInput = new MemoryStream(buffer);
using (System.IO.Stream source = streamInput)
{
byte[] bufferRead = new byte[frameCount];
int bytesRead;
while ((bytesRead = source.Read(bufferRead, 0, bufferRead.Length)) > 0)
{
if (recognizer.AcceptWaveform(bufferRead, bytesRead))
{
string result = recognizer.Result();
Log.VerboseFormat("Result: {0}", result);
if (SpeechRecognizedEvent != null)
{
SpeechRecognizedEvent(result);
}
}
}
}
}
catch (Exception ex)
{
Log.Error(ex);
}
return StreamCallbackResult.Continue;
}
```
EDIT:
Half an hour later, I tried option #3: vosk with NAudio and it not only generated the correct audio file (no wonders here because that use case worked 12 years ago), but also generated the correct text that I output to screen... Problem Solved!
Still, I wondering why vosk initialization takes 15 seconds... probably due to model size of 1GB+ (?)
Guía de contribución
No hay ninguna guía de contribución indexada para este repositorio
Evaluación
Este issue todavía no se ha evaluado.