Questions/change request proposal about decoding output format (result, partialResult, finalResult)
- Lingua principale
- Jupyter Notebook
- Stelle
- 15.1k
- Fork
- 1.8k
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Hi Nicolay
following you comment on closed issue: https://github.com/alphacep/vosk-api/issues/590#issuecomment-863065813, you stated that:
> 1. AcceptWaveform returns true when silence is detected and you can retrieve the result with Result().
> 2. If silence is not detected you can retrieve PartialResult() only.
> 3. FinalResult means the stream is ended, you flush the buffers and retrieve remaining result.
Now, I tested above methods processing a speech file containing some sentences separated by "silences":
`one two three` silence `four five six seven eight` silence `nine zero one two three stop`
So the audio contains "three" different sentences (considering that the silence / end of audio terminate a sentence).
- `one two three`
- `four five six seven eight`
- `nine zero one two three stop`
Here the audio: https://github.com/solyarisoftware/voskJs/blob/master/audio/sentencesWithSilences.wav
If I run my voskjs (nodejs cli wrapper to vosk-api) I get:
```
$ voskjs --audio=audio/sentencesWithSilences.wav --model=models/vosk-model-small-en-us-0.15
model directory : models/vosk-model-small-en-us-0.15
speech file name : audio/sentencesWithSilences.wav
grammar : not specified. Default: NO
sample rate : not specified. Default: 16000
max alternatives : undefined
text only / JSON : JSON
Vosk debug level : -1
load model latency : 285ms
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: 'one' }
partialResult { partial: 'one' }
partialResult { partial: 'one two' }
partialResult { partial: 'one two' }
partialResult { partial: 'one two three' }
endOfSpeechResult {
result: [
{ conf: 1, end: 1.95, start: 1.41, word: 'one' },
{ conf: 1, end: 2.7, start: 2.28, word: 'two' },
{ conf: 1, end: 3.96, start: 3.45, word: 'three' }
],
text: 'one two three'
}
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: 'for' }
partialResult { partial: 'four five six' }
partialResult { partial: 'four five six' }
partialResult { partial: 'four five six seven' }
partialResult { partial: 'four five six seven' }
partialResult { partial: 'four five six seven eight' }
partialResult { partial: 'four five six seven eight' }
partialResult { partial: 'four five six seven eight' }
endOfSpeechResult {
result: [
{ conf: 1, end: 7.44, start: 6.78, word: 'four' },
{ conf: 1, end: 8.28, start: 7.68, word: 'five' },
{ conf: 1, end: 9.18, start: 8.37, word: 'six' },
{ conf: 1, end: 9.96, start: 9.24, word: 'seven' },
{ conf: 1, end: 10.8, start: 10.17, word: 'eight' }
],
text: 'four five six seven eight'
}
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: 'nine' }
partialResult { partial: 'nine zero' }
partialResult { partial: 'nine zero' }
partialResult { partial: 'nine zero' }
partialResult { partial: 'nine zero one two' }
partialResult { partial: 'nine zero one two' }
partialResult { partial: 'nine zero one two three' }
partialResult { partial: 'nine zero one two three' }
partialResult { partial: 'nine zero one two three stop' }
partialResult { partial: 'nine zero one two three stop' }
endOfSpeechResult {
result: [
{ conf: 1, end: 13.74, start: 13.05, word: 'nine' },
{ conf: 1, end: 14.64, start: 13.95, word: 'zero' },
{ conf: 1, end: 15.42, start: 14.79, word: 'one' },
{ conf: 1, end: 16.17, start: 15.69, word: 'two' },
{ conf: 1, end: 16.98, start: 16.47, word: 'three' },
{ conf: 1, end: 17.85, start: 17.28, word: 'stop' }
],
text: 'nine zero one two three stop'
}
partialResult { partial: '' }
finalResult { text: '' }
transcript latency : 1880ms
```
Vosk correctly transcript all sentences, detecting the end_of_speech, but I have some related questions and proposals:
### Question 1
How is calculated the `end_of_speech`?
I searched in un the C code and It seems to me that's related to `Endpointdetected` function:
```
$ rg EndpointDetected
src/kaldi_recognizer.cc
307: if (decoder_->EndpointDetected(model_->endpoint_config_)) {
```
But I didn't find how is defined/how it works EndpontDetected. Anyway my question is about how is calculated the "silence". Maybe the calculation is about the elapsed time (in milliseconds, call it `SilenceDuration`) when volume (RMS) is under a threshold value (call it `TresholdVolume`) ?
If so, can I configure/set `TresholdVolume` and/or `SilenceDuration` in the vosk-api ? It is possible to allow vock-api user to configure/tune the silence attributes?
### Question 2
Consider the first sentence `one two three` partial results:
```
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: '' }
partialResult { partial: 'one' }
partialResult { partial: 'one' }
partialResult { partial: 'one two' }
partialResult { partial: 'one two' }
partialResult { partial: 'one two three' }
```
Why I get 5 times a void string partial result?
Ok, I can discard these void strings in my application, but why these void results are emitted? There is something underneath I do not see? If no, my proposal is to do not emit these void strings.
### Question 3
As far as I understand the `finalResult` attribute run as expected (collecting the full transcript of the audio) just if the audio do not contains multi sentences (sentences separated by silences) if the audio do not contains end_of_speeches.
Otherwise the finalResult does not contain nothing (that result in the reported example). That seems to me a possible functional "bug". I would expect instead/I'd propose the finalResult to be a list of sentences (partialresult); something like this:
```
[
{
words: [
{ conf: 1, end: 1.95, start: 1.41, word: 'one' },
{ conf: 1, end: 2.7, start: 2.28, word: 'two' },
{ conf: 1, end: 3.96, start: 3.45, word: 'three' }
],
text: 'one two three'
},
{
words: [
{ conf: 1, end: 7.44, start: 6.78, word: 'four' },
{ conf: 1, end: 8.28, start: 7.68, word: 'five' },
{ conf: 1, end: 9.18, start: 8.37, word: 'six' },
{ conf: 1, end: 9.96, start: 9.24, word: 'seven' },
{ conf: 1, end: 10.8, start: 10.17, word: 'eight' }
],
text: 'four five six seven eight'
},
{
words: [
{ conf: 1, end: 13.74, start: 13.05, word: 'nine' },
{ conf: 1, end: 14.64, start: 13.95, word: 'zero' },
{ conf: 1, end: 15.42, start: 14.79, word: 'one' },
{ conf: 1, end: 16.17, start: 15.69, word: 'two' },
{ conf: 1, end: 16.98, start: 16.47, word: 'three' },
{ conf: 1, end: 17.85, start: 17.28, word: 'stop' }
],
text: 'nine zero one two three stop'
}
]
```
What do you think?
Thanks for your patience!
Giorgio
Guida per i contributori
Nessuna guida per i contributori indicizzata per questo repository
Valutazione
Questa issue non è ancora stata valutata.