Request for Guidance on Tamil Voice Cloning – Training Data, Metadata & Multi-Speaker Support

Checks

• This template is only for research question, not usage problems, feature requests or bug reports.
• I have thoroughly reviewed the project documentation and read the related paper(s).
• I have searched for existing issues, including closed ones, no similar questions.
• I am using English to submit this issue to facilitate community communication.

Question details

Hello there,

I am currently working on a Tamil TTS project with the goal of achieving high-quality voice cloning using only training text. I have a few questions and would really appreciate your guidance and support:

Data Requirements: To achieve accurate and natural-sounding voice cloning in Tamil, how many hours of audio data would you recommend?

Emotion Metadata: Does the training process support or require emotional classification in the metadata (e.g., happy, sad), or is neutral speech sufficient?

Generalization Issue: In a previous experiment, I used 200 hours of audio. The model performed well when reproducing the trained voice, but it did not generalize effectively to other voices.

Configuration used: batch size = 700, 20 epochs, 1.3M training steps.

Given this setup, what would you suggest to enable voice cloning for new speakers?

I’d be grateful for any insights or suggestions. If needed, I’d also be happy to arrange a call or meeting to discuss this further.

@SWivid

The issue names no files, tests, or code entry points. Start with the project documentation and related papers referenced by the author, then examine the training configuration described in the issue. A concrete, scoped change and acceptance criteria would be needed before implementation can be considered done.