EveryVoiceTTS / EveryVoiceTTS/EveryVoice

Preprocessing seems to require a specific order, but cross-model preprocessing should be improved

Open
#858 4 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
45
Forks
4
Avg merge
1d 8h
Merged PRs (30d)
14

Description

### Bug description

StyleTTS2 preprocessing only requires text, audio (and OOD text)
FastSpeech2 preprocessing requires text, audio, spec, pitch, energy, attn and stats.

I ran FastSpeech2 preprocessing first, but then realized I needed to re-run preprocessing with StyleTTS2 to get the processed OOD text. We should be able to either preprocess OOD text separately and individually. Or, we should be able to do preprocessing for both. Similarly, re-running the pitch and energy stats for FastSpeech2 after preprocessing is complete didn't work until I re-ran processing altogether. This is not very ergonomic.

Contributor guide

Open the contributing guide

Research direction

Trace the StyleTTS2 and FastSpeech2 preprocessing workflows and reproduce the order-dependent runs described in the issue. Identify how OOD text, pitch, energy, and related artifacts are coupled; done means these preprocessing steps can be run independently or together without rerunning unrelated processing.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.