alphacep / alphacep/vosk-api

How to get more "love" word into VOSK?

Offen
#1,184 5 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Vorherrschende Sprache
Jupyter Notebook
Sterne
15.1k
Forks
1.8k
PR-Merge-Kennzahlen
Keine gemergten PRs in 30 T.

Beschreibung

Hi, weird title I know. I'm trying to use VOSK on some tennis recordings where scores like "fifteen love" comes up. Sadly the model I am using is not great at picking up the "love" element, whether before or after. I have read that there are options to enhance word identification however I don't know if this will work ( and whilst there are some docs on how to adjust this, it looked a little beyond my capability, so I am posting this question first to get feedback).

The reason why I think this will NOT work is because I have built 2 VOSK models and simply changed the vocab. In the second, "love" is almost the only word in the custom dictionary, and there I can see that where this is picked up (timestamp) is in the middle of the prior word (ie fifteen).

Below my screen shots:
Full grammer model output - fifteen is picked up between 23,82 & 24.208
![image](https://user-images.githubusercontent.com/13690904/198883922-0ed4c4bb-613e-4068-a1b0-0c876e16320d.png)

Love Grammar model output - love is picked up at 24.15 (ie in the middle of the above)
![image](https://user-images.githubusercontent.com/13690904/198883956-5b20833a-df9a-4bdf-8ce5-c243d5f53fa9.png)

My planned approach is to run the model twice, each time outputting the word and the elements of result into a table to be able to construct the phrase. The only challenge here is that it double the run time.

My question is whether the enhancements of the language / specific grammar / increased probabilities will help resolve this issue. I have the same issue with "fifteen all" and there my solution doesn't work as "all" or a soundalike doesn't get picked up by a separate model.

I can provide example sound clips if that helps you help me.

My code:
```
def text_from_audio_v3( path, file, lang, location):
os.chdir("D:/OneDrive/DataSci/Tennis/02_Preprocessing/Voice/VSOK/" + location)
from pydub import AudioSegment
# wf = AudioSegment.from_file(path + file)
wf = wave.open(path + file, "rb")
model = Model("model")

if lang == "English":
# rec = KaldiRecognizer(model, wf.getframerate(), '["love", "fifteen","thirty","forty","deuce", "mistake","winner","double", "fault", "second", "serve", "let","advantage", "my"]')
# rec = KaldiRecognizer(model, wf.getframerate(),
# '["love", "fifteen","thirty","forty","deuce", "mistake","winner","double fault", "second serve", "let"," my advantage","your advantage","all","game"]')
rec = KaldiRecognizer(model, wf.getframerate(),
'["fifteen","love", "thirty","or","all", "forty", "deuce","juice", "game","mistake","winner","forced","second", "my advantage","your advantage", "his advantage" ]')
# '["love","all", "or" ]')
# '["love", "fifteen","thirty","forty","deuce", "mistake","winner","forced","second", "my advantage","your advantage", "his advantage","all","each","game" ]')
# rec = KaldiRecognizer(model, wf.getframerate(), '["eins", "null", "fehler"]')
# rec = KaldiRecognizer(model, wf.getframerate(), '["second", "serve", "love", "fifteen", "mistake", "thirty", "winner"]')
results = []
while True:
data = wf.readframes(4000)
if len(data) == 0:
break
if rec.AcceptWaveform(data):
print(rec.Result())

results.append(json.loads(rec.FinalResult())['text'])
pprint.pprint(results)
return results
```

Beitragsleitfaden

Für dieses Repository ist kein Beitragsleitfaden indexiert

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.