alphacep / alphacep/vosk-api

Add speaker vector to nbest

Ouverte
#1,199 5 commentaires 0 réactions 0 personnes assignées Voir sur GitHub
Langage dominant
Jupyter Notebook
Étoiles
15.1k
Forks
1.8k
Métriques de merge des PR
Aucune PR mergée en 30 j

Description

Hello!

I have modified the Unity sample to do speaker diarization, and I can't seem to get the diarization information coming through.

```
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{

var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);

//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);

_recognizerReady = true;

Debug.Log("Recognizer ready");
}
```

I am on an M1 mac, and to get it working at all I needed to take your m1 build from your Python distribution and add it to the Unity project.

I also note that your code for pulling the PCM audio from Unity doesn't seem to work, it just produces gibberish, for now I just read in some PCM shorts from a test wave file and that works:

```
// stream bytes from output.wav in StreamingAssets folder
var stream = new FileStream(Path.Combine(Application.streamingAssetsPath, "output.wav"), FileMode.Open);
// this is a PCM wave file, parse it in read the audio data in shorts
var reader = new WaveFileReader(stream);
byte[] buffer = new byte[reader.Length / 2];
reader.Read(buffer, 0, buffer.Length);
// feed the audio data to the recognizer 4000 samples at a time

// get a byte[] of 4000 samples buffer, i, length
byte[] buffer4000 = new byte[4000];

for (int i = 0; i < buffer.Length; i += 4000)
{

Array.Copy(buffer, i, buffer4000, 0, 4000);


if (_recognizer.AcceptWaveform(buffer4000, buffer4000.Length))
{
var result = _recognizer.FinalResult();
//_recognizer.Reset();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}
else
{
var result = _recognizer.PartialResult();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}

await Task.Delay(100);
}
```

I make some minor modifications:

```
private async Task ThreadedWork()
{
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{

var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);

//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);

_recognizerReady = true;

Debug.Log("Recognizer ready");
}
```

There is no documentation on this but I I think this is right i.e.

* make a SpkModel pointing to the decompressed path of the speaker model, I am using `vosk-model-spk-0.4`
* Make a VoskRecognizer model pointing to a normal english model, I am using `vosk-model-en-us-0.22`

It runs, and produces English transcription, but I don't see any speaker embeddings on the final transcript

Partials are coming through like this:

```
{
"partial" : "just because you can execute a program on the computer"
}
```

And finals are coming through like this:

```
{
"alternatives" : [{
"confidence" : 677.793457,
"text" : "just because you can execute a program on the computer to perform the task does not mean that the system you've created understands the dos"
}]
}
```

I know I am probably being stupid here, what am I doing wrong?!

Help! Thanks!

Guide de contribution

Aucun guide de contribution indexé pour ce dépôt

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.