alphacep / alphacep/vosk-api

How to get more "love" word into VOSK?

Open
#1,184 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
15.1k
Forks
1.8k
PR merge metrics
No merged PRs in 30d

Description

Hi, weird title I know. I'm trying to use VOSK on some tennis recordings where scores like "fifteen love" comes up. Sadly the model I am using is not great at picking up the "love" element, whether before or after. I have read that there are options to enhance word identification however I don't know if this will work ( and whilst there are some docs on how to adjust this, it looked a little beyond my capability, so I am posting this question first to get feedback).

The reason why I think this will NOT work is because I have built 2 VOSK models and simply changed the vocab. In the second, "love" is almost the only word in the custom dictionary, and there I can see that where this is picked up (timestamp) is in the middle of the prior word (ie fifteen).

Below my screen shots:
Full grammer model output - fifteen is picked up between 23,82 & 24.208
![image](https://user-images.githubusercontent.com/13690904/198883922-0ed4c4bb-613e-4068-a1b0-0c876e16320d.png)

Love Grammar model output - love is picked up at 24.15 (ie in the middle of the above)
![image](https://user-images.githubusercontent.com/13690904/198883956-5b20833a-df9a-4bdf-8ce5-c243d5f53fa9.png)

My planned approach is to run the model twice, each time outputting the word and the elements of result into a table to be able to construct the phrase. The only challenge here is that it double the run time.

My question is whether the enhancements of the language / specific grammar / increased probabilities will help resolve this issue. I have the same issue with "fifteen all" and there my solution doesn't work as "all" or a soundalike doesn't get picked up by a separate model.

I can provide example sound clips if that helps you help me.

My code:
```
def text_from_audio_v3( path, file, lang, location):
os.chdir("D:/OneDrive/DataSci/Tennis/02_Preprocessing/Voice/VSOK/" + location)
from pydub import AudioSegment
# wf = AudioSegment.from_file(path + file)
wf = wave.open(path + file, "rb")
model = Model("model")

if lang == "English":
# rec = KaldiRecognizer(model, wf.getframerate(), '["love", "fifteen","thirty","forty","deuce", "mistake","winner","double", "fault", "second", "serve", "let","advantage", "my"]')
# rec = KaldiRecognizer(model, wf.getframerate(),
# '["love", "fifteen","thirty","forty","deuce", "mistake","winner","double fault", "second serve", "let"," my advantage","your advantage","all","game"]')
rec = KaldiRecognizer(model, wf.getframerate(),
'["fifteen","love", "thirty","or","all", "forty", "deuce","juice", "game","mistake","winner","forced","second", "my advantage","your advantage", "his advantage" ]')
# '["love","all", "or" ]')
# '["love", "fifteen","thirty","forty","deuce", "mistake","winner","forced","second", "my advantage","your advantage", "his advantage","all","each","game" ]')
# rec = KaldiRecognizer(model, wf.getframerate(), '["eins", "null", "fehler"]')
# rec = KaldiRecognizer(model, wf.getframerate(), '["second", "serve", "love", "fifteen", "mistake", "thirty", "winner"]')
results = []
while True:
data = wf.readframes(4000)
if len(data) == 0:
break
if rec.AcceptWaveform(data):
print(rec.Result())

results.append(json.loads(rec.FinalResult())['text'])
pprint.pprint(results)
return results
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.