EveryVoiceTTS / EveryVoiceTTS/EveryVoice
everyvoice segment is not robust to alignment mismatches
- Dominant language
- Python
- Stars
- 45
- Forks
- 4
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 14
Description
When `compute_alignment()` in `wav2vec2aligner/aligner/heavy.py` does not successfully compute alignments for each word, we get some entries in `word_hash` that are still CompositeTransductionGraphs, causing an exception dump.
I added `rich.print(word_hash)` before line 142 to get this output:
```
$ echo "For two more years these high figures were steadily maintained, and in 1803 the total rose to 710." > foo.txt
$ everyvoice segment foo.txt wavs/LJ002-0011.wav
loading model...
resampling audio from 22050 to 16000
processing text
performing alignment
{
's0w0': For (1.00): [ 1, 5),
's0w1': two (1.00): [ 11, 16),
's0w2': more (1.00): [ 24, 31),
's0w3': years (1.00): [ 40, 60),
's0w4': these (0.96): [ 90, 101),
's0w5': high (0.99): [ 108, 116),
's0w6': figures (1.00): [ 124, 144),
's0w7': were (0.97): [ 154, 160),
's0w8': steadily (0.98): [ 164, 184),
's0w9': maintained, (0.98): [ 189, 221),
's0w10': and (0.89): [ 255, 263),
's0w11': in (0.96): [ 278, 297),
's0w12': 1803 (0.88): [ 313, 477),
's0w13': {'text': },
's0w14': {'text': },
's0w15': {'text': },
's0w16': {'text': },
's0w17': {'text': }
}
╭──────────────────────────── Traceback (most recent call last) ────────────────────────────╮
│ /gpfs/fs3c/nrc/ict/portage/u/joa125/EveryVoice/EveryVoice/everyvoice/model/aligner/wav2ve │
│ c2aligner/aligner/cli.py:68 in align_single │
│ │
│ 65 │ print("performing alignment") │
│ 66 │ from .heavy import align_speech_file │
│ 67 │ │
│ ❱ 68 │ characters, words, sentences, num_frames = align_speech_file( │
│ 69 │ │ wav, text_hash, model, labels, word_padding, sentence_padding │
│ 70 │ ) │
│ 71 │ print("creating textgrid") │
│ │
│ /gpfs/fs3c/nrc/ict/portage/u/joa125/EveryVoice/EveryVoice/everyvoice/model/aligner/wav2ve │
│ c2aligner/aligner/heavy.py:32 in align_speech_file │
│ │
│ 29 │ audio, text_hash, model, labels_dictionary, word_padding, sentence_padding │
│ 30 ): │
│ 31 │ emission = get_emission(model, audio.to(DEVICE)) │
│ ❱ 32 │ segments, words, sentences = compute_alignments( │
│ 33 │ │ text_hash, │
│ 34 │ │ labels_dictionary, │
│ 35 │ │ emission, │
│ │
│ /gpfs/fs3c/nrc/ict/portage/u/joa125/EveryVoice/EveryVoice/everyvoice/model/aligner/wav2ve │
│ c2aligner/aligner/heavy.py:147 in compute_alignments │
│ │
│ 144 │ │ for w_k, w_v in word_hash.items(): │
│ 145 │ │ │ if sentence == re.match(key_pattern, w_k).group(1): │
│ 146 │ │ │ │ print(w_v) │
│ ❱ 147 │ │ │ │ scores.append(w_v.score) │
│ 148 │ │ │ │ if start is None: │
│ 149 │ │ │ │ │ start = w_v.start │
│ 150 │ │ │ │ end = w_v.end │
╰───────────────────────────────────────────────────────────────────────────────────────────╯
AttributeError: 'dict' object has no attribute 'score'
```
Contributor guide
Assessment
This issue has not been assessed yet.