alphacep / alphacep/vosk-api

Add speaker vector to nbest

Offen
#1,199 5 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Jupyter Notebook
Sterne
15.1k
Forks
1.8k
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hello!

I have modified the Unity sample to do speaker diarization, and I can't seem to get the diarization information coming through.

```
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{

var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);

//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);

_recognizerReady = true;

Debug.Log("Recognizer ready");
}
```

I am on an M1 mac, and to get it working at all I needed to take your m1 build from your Python distribution and add it to the Unity project.

I also note that your code for pulling the PCM audio from Unity doesn't seem to work, it just produces gibberish, for now I just read in some PCM shorts from a test wave file and that works:

```
// stream bytes from output.wav in StreamingAssets folder
var stream = new FileStream(Path.Combine(Application.streamingAssetsPath, "output.wav"), FileMode.Open);
// this is a PCM wave file, parse it in read the audio data in shorts
var reader = new WaveFileReader(stream);
byte[] buffer = new byte[reader.Length / 2];
reader.Read(buffer, 0, buffer.Length);
// feed the audio data to the recognizer 4000 samples at a time

// get a byte[] of 4000 samples buffer, i, length
byte[] buffer4000 = new byte[4000];

for (int i = 0; i < buffer.Length; i += 4000)
{

Array.Copy(buffer, i, buffer4000, 0, 4000);


if (_recognizer.AcceptWaveform(buffer4000, buffer4000.Length))
{
var result = _recognizer.FinalResult();
//_recognizer.Reset();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}
else
{
var result = _recognizer.PartialResult();
Debug.Log(result);
_threadedResultQueue.Enqueue(result);
}

await Task.Delay(100);
}
```

I make some minor modifications:

```
private async Task ThreadedWork()
{
voskRecognizerCreateMarker.Begin();
if (!_recognizerReady)
{

var spkModel = new SpkModel(_decompressedModelPath);
_recognizer = new VoskRecognizer(_model, 16000.0f);
_recognizer.SetSpkModel(spkModel);

//
_recognizer.SetMaxAlternatives(1);
//_recognizer.SetWords(true);

_recognizerReady = true;

Debug.Log("Recognizer ready");
}
```

There is no documentation on this but I I think this is right i.e.

* make a SpkModel pointing to the decompressed path of the speaker model, I am using `vosk-model-spk-0.4`
* Make a VoskRecognizer model pointing to a normal english model, I am using `vosk-model-en-us-0.22`

It runs, and produces English transcription, but I don't see any speaker embeddings on the final transcript

Partials are coming through like this:

```
{
"partial" : "just because you can execute a program on the computer"
}
```

And finals are coming through like this:

```
{
"alternatives" : [{
"confidence" : 677.793457,
"text" : "just because you can execute a program on the computer to perform the task does not mean that the system you've created understands the dos"
}]
}
```

I know I am probably being stupid here, what am I doing wrong?!

Help! Thanks!

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.