Add speaker vector to nbest
- Langage dominant
- Jupyter Notebook
- Étoiles
- 15.1k
- Forks
- 1.8k
- Métriques de merge des PR
- Aucune PR mergée en 30 j
Description
Hello!
I have modified the Unity sample to do speaker diarization, and I can't seem to get the diarization information coming through.
```
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{
var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);
//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);
_recognizerReady = true;
Debug.Log("Recognizer ready");
}
```
I am on an M1 mac, and to get it working at all I needed to take your m1 build from your Python distribution and add it to the Unity project.
I also note that your code for pulling the PCM audio from Unity doesn't seem to work, it just produces gibberish, for now I just read in some PCM shorts from a test wave file and that works:
```
// stream bytes from output.wav in StreamingAssets folder
var stream = new FileStream(Path.Combine(Application.streamingAssetsPath, "output.wav"), FileMode.Open);
// this is a PCM wave file, parse it in read the audio data in shorts
var reader = new WaveFileReader(stream);
byte[] buffer = new byte[reader.Length / 2];
reader.Read(buffer, 0, buffer.Length);
// feed the audio data to the recognizer 4000 samples at a time
// get a byte[] of 4000 samples buffer, i, length
byte[] buffer4000 = new byte[4000];
for (int i = 0; i < buffer.Length; i += 4000)
{
Array.Copy(buffer, i, buffer4000, 0, 4000);
if (_recognizer.AcceptWaveform(buffer4000, buffer4000.Length))
{
var result = _recognizer.FinalResult();
//_recognizer.Reset();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}
else
{
var result = _recognizer.PartialResult();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}
await Task.Delay(100);
}
```
I make some minor modifications:
```
private async Task ThreadedWork()
{
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{
var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);
//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);
_recognizerReady = true;
Debug.Log("Recognizer ready");
}
```
There is no documentation on this but I I think this is right i.e.
* make a SpkModel pointing to the decompressed path of the speaker model, I am using `vosk-model-spk-0.4`
* Make a VoskRecognizer model pointing to a normal english model, I am using `vosk-model-en-us-0.22`
It runs, and produces English transcription, but I don't see any speaker embeddings on the final transcript
Partials are coming through like this:
```
{
"partial" : "just because you can execute a program on the computer"
}
```
And finals are coming through like this:
```
{
"alternatives" : [{
"confidence" : 677.793457,
"text" : "just because you can execute a program on the computer to perform the task does not mean that the system you've created understands the dos"
}]
}
```
I know I am probably being stupid here, what am I doing wrong?!
Help! Thanks!
Guide de contribution
Aucun guide de contribution indexé pour ce dépôt
Évaluation
Cette issue n'a pas encore été évaluée.