Persian TTS Inference Issues: Repeated Words, Missing Words, and Inconsistent Pronunciation

I fine-tuned both the VoxCPM 1.5B and VoxCPM 0.5B models on a Persian-language dataset containing more than 150 hours of audio.

The models were trained for approximately:

• VoxCPM 1.5B: 90,000 steps
• VoxCPM 0.5B: 70,000 steps

The overall audio quality of the generated speech is very good. However, I consistently encounter several problems during inference, particularly with Persian text.

1. Repetition of a part of the sentence

Sometimes the model repeats a part of the input sentence during generation.

For example, the following English example demonstrates the problem:

Input text:

On a bright summer afternoon, while the golden sunlight poured over the sprawling park and the gentle breeze rustled the leaves of towering trees, families strolled along winding paths, children laughed as they played near the sparkling fountains.

Generated output:

On a bright summer afternoon, while the golden sunlight poured over the sprawling park and the gentle breeze rustled the leaves of towering trees, families strolled along winding paths, children laughed as they played near the sparkling fountains.

The bold part is repeated in the generated speech.

This English example is only provided to demonstrate the behavior. The actual problem occurs with Persian text as well.

2. Missing words at the beginning of the sentence

In some cases, one or more words at the beginning of the input sentence are not pronounced in the generated audio.

For example:

Input text:

On a bright summer afternoon, while the golden sunlight poured over the sprawling park and the gentle breeze rustled the leaves of towering trees, families strolled along winding paths, children laughed as they played near the sparkling fountains.

Generated output:

On a bright summer afternoon, while the golden sunlight poured over the sprawling park and the gentle breeze rustled the leaves of towering trees, families strolled along winding paths, children laughed as they played near the sparkling fountains.

In this example, the bold part represents words that are missing from the generated speech.

Again, this English example is only intended to illustrate the issue. The same problem occurs with Persian text.

3. Inconsistent pronunciation between inference runs

Another issue is that the pronunciation of some words is not consistent.

For the same input sentence, running inference multiple times can sometimes produce different pronunciations for certain words.

For example, a particular Persian word may be pronounced correctly in one inference run but incorrectly or differently in another run, even though:

• The input text is exactly the same.
• The trained model is exactly the same.
• The inference configuration is unchanged.

This makes the generated speech somewhat nondeterministic in terms of pronunciation.

Additional Information

Despite these problems, the overall quality of the generated Persian speech is very good. The main issues are:

1. Occasional repetition of phrases or parts of sentences
2. Occasional omission of words, especially at the beginning of a sentence
3. Inconsistent pronunciation of some words across different inference runs

These issues occur in both the VoxCPM 1.5B and 0.5B models that I fine-tuned.

Questions

I would appreciate any suggestions regarding the possible causes and solutions for these issues.

In particular:

• Could these problems be related to the training data, tokenization, or text normalization for Persian?
• Could they be caused by the duration prediction, alignment, or autoregressive generation process?
• Are there recommended inference parameters or decoding strategies to reduce repetition and missing words?
• Is there any recommended configuration for fine-tuning VoxCPM specifically for Persian or other languages with similar tokenization characteristics?
• Is there a recommended way to make pronunciation more consistent between different inference runs?

Any guidance on how to diagnose or fix these issues would be greatly appreciated.

real data:

text:
"یکی از مهم ترین چالش ها، تلفظ صحیح کلمات است، که به شدت به کیفیت داده های آموزشی و توانایی مدل در یادگیری وابسته است. اگر داده ها تنوع و کیفیت کافی نداشته باشند، خروجی مدل دچار خطاهای تلفظی می‌شود."

output:

data ground truth:
ground truth.mp3

output full fine tune voxcpm 0.5 B:
voxcom2.wav
voxcpm1.wav
voxcpm3.wav

Start by reproducing the attached Persian example with the same fine-tuned VoxCPM 0.5B and 1.5B models, comparing repeated, missing, and variably pronounced words across runs. Inspect the training data, Persian tokenization and normalization, alignment or duration behavior, and inference configuration; done means identifying a reproducible cause and documenting a validated mitigation.