bytedance / bytedance/MegaTTS3

Training collapses when using random single-anchor retention(sparse alignment) in expand_states (outputs pure noise)

Open
#145 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
6.1k
Forks
474
PR merge metrics
No merged PRs in 30d

Description

Hi MegaTTS3 team, thanks for the great work!
I have a question about training stability when modifying `expand_states` anchor retention.

## What I changed
During training, in `expand_states`, I tried a random strategy that keeps only **one anchor** (randomly selected) per sample/step.

## Problem
With this setting, training seems to collapse:
- model does not learn meaningful content
- generated audio is mostly noisy / contentless
- loss behavior does not indicate normal convergence

## My question
Is there any known trick for random anchor retention in MegaTTS3 training?
Specifically:
1. Is keeping only one anchor too aggressive at early training?
2. Is a curriculum strategy recommended, e.g.:
- early stage: keep all or most anchors
- later stage: gradually reduce the number of anchors
3. Are there recommended sampling constraints (minimum anchors, position constraints, ratio schedule)?
4. Are there any related hyperparameters that should be adjusted together (LR, warmup, loss weights, etc.)?

Thanks in advance!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the training path around `expand_states` and reproduce the reported setup that retains one randomly selected anchor per sample or step. Compare training behavior with all or most anchors retained, and document whether a curriculum, sampling constraints, or related hyperparameter changes prevent noisy, contentless output.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.