alphacep / alphacep/vosk-api

Add speaker vector to nbest

Abierto
#1,199 5 comentarios 0 reacciones 0 asignados Ver en GitHub
Lenguaje dominante
Jupyter Notebook
Estrellas
15.1k
Forks
1.8k
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

Hello!

I have modified the Unity sample to do speaker diarization, and I can't seem to get the diarization information coming through.

```
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{

var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);

//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);

_recognizerReady = true;

Debug.Log("Recognizer ready");
}
```

I am on an M1 mac, and to get it working at all I needed to take your m1 build from your Python distribution and add it to the Unity project.

I also note that your code for pulling the PCM audio from Unity doesn't seem to work, it just produces gibberish, for now I just read in some PCM shorts from a test wave file and that works:

```
// stream bytes from output.wav in StreamingAssets folder
var stream = new FileStream(Path.Combine(Application.streamingAssetsPath, "output.wav"), FileMode.Open);
// this is a PCM wave file, parse it in read the audio data in shorts
var reader = new WaveFileReader(stream);
byte[] buffer = new byte[reader.Length / 2];
reader.Read(buffer, 0, buffer.Length);
// feed the audio data to the recognizer 4000 samples at a time

// get a byte[] of 4000 samples buffer, i, length
byte[] buffer4000 = new byte[4000];

for (int i = 0; i < buffer.Length; i += 4000)
{

Array.Copy(buffer, i, buffer4000, 0, 4000);


if (_recognizer.AcceptWaveform(buffer4000, buffer4000.Length))
{
var result = _recognizer.FinalResult();
//_recognizer.Reset();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}
else
{
var result = _recognizer.PartialResult();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}

await Task.Delay(100);
}
```

I make some minor modifications:

```
private async Task ThreadedWork()
{
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{

var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);

//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);

_recognizerReady = true;

Debug.Log("Recognizer ready");
}
```

There is no documentation on this but I I think this is right i.e.

* make a SpkModel pointing to the decompressed path of the speaker model, I am using `vosk-model-spk-0.4`
* Make a VoskRecognizer model pointing to a normal english model, I am using `vosk-model-en-us-0.22`

It runs, and produces English transcription, but I don't see any speaker embeddings on the final transcript

Partials are coming through like this:

```
{
"partial" : "just because you can execute a program on the computer"
}
```

And finals are coming through like this:

```
{
"alternatives" : [{
"confidence" : 677.793457,
"text" : "just because you can execute a program on the computer to perform the task does not mean that the system you've created understands the dos"
}]
}
```

I know I am probably being stupid here, what am I doing wrong?!

Help! Thanks!

Guía de contribución

No hay ninguna guía de contribución indexada para este repositorio

Evaluación

Este issue todavía no se ha evaluado.

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.