bytedance / bytedance/MegaTTS3
Part of sentences are stripped from the audio output.
- Dominant language
- Python
- Stars
- 6.1k
- Forks
- 474
- PR merge metrics
- No merged PRs in 30d
Description
I'm executing this CMD with the default English_prompt.wav
`
python ./tts/infer_cli.py --input_wav ./assets/English_prompt.wav --input_text "As his long promised tariff threat turned into reality this week, top human advisers began fielding a wave of calls from business leaders, particularly in the automotive sector, along with lawmakers who were sounding the alarm." --output_dir ./gen --p_w 2.0 --t_w 3.0
`
the result is the part of text are stripped "...along with lawmakers who were sounding the alarm"

then I run again with my own local voice
`
python ./tts/infer_cli.py --input_wav ./assets/sample_zzz_en.wav --input_text "As his long promised tariff threat turned into reality this week, top human advisers began fielding a wave of calls from business leaders, particularly in the automotive sector, along with lawmakers who were sounding the alarm." --output_dir ./gen --p_w 2.0 --t_w 3.0
`
now the first part of the text are stripped "As his long promised tariff threat turned into reality this week,.."

here's the generated audio sample
https://drive.google.com/file/d/1m7tB2YqaEchS1Qskp-7pmH_JDpEEC_Nw/view?usp=sharing
as for the log for the later cmd:
~~~
| loaded 'dur_model' from './checkpoints\duration_lm/model_only_last.ckpt'.
| Missing keys: 0, Unexpected keys: 0
| loaded 'dit' from './checkpoints\diffusion_transformer/model_only_last.ckpt'.
| Missing keys: 0, Unexpected keys: 9
| loaded 'model' from './checkpoints\aligner_lm/model_only_last.ckpt'.
| Missing keys: 0, Unexpected keys: 0
D:\DEVSDK\anaconda3\envs\megatts3-env\lib\site-packages\torch\nn\utils\weight_norm.py:143: FutureWarning: `torch.nn.utils.weight_norm` is deprecated in favor of `torch.nn.utils.parametrizations.weight_norm`.
WeightNorm.apply(module, name, dim)
| loaded 'model_gen' from './checkpoints\wavvae/decoder.ckpt'.
| Missing keys: 74, Unexpected keys: 0
2025-04-12 18:00:00,588 WETEXT INFO found existing fst: D:\DEVSDK\anaconda3\envs\megatts3-env\lib\site-packages\tn\zh_tn_tagger.fst
2025-04-12 18:00:00,588 WETEXT INFO D:\DEVSDK\anaconda3\envs\megatts3-env\lib\site-packages\tn\zh_tn_verbalizer.fst
2025-04-12 18:00:00,588 WETEXT INFO skip building fst for zh_normalizer ...
2025-04-12 18:00:01,268 WETEXT INFO found existing fst: D:\DEVSDK\anaconda3\envs\megatts3-env\lib\site-packages\tn\en_tn_tagger.fst
2025-04-12 18:00:01,268 WETEXT INFO D:\DEVSDK\anaconda3\envs\megatts3-env\lib\site-packages\tn\en_tn_verbalizer.fst
2025-04-12 18:00:01,268 WETEXT INFO skip building fst for en_normalizer ...
| Start processing ./assets/sample_zzz_en.wav+As his long promised tariff threat turned into reality this week, top human advisers began fielding a wave of calls from business leaders, particularly in the automotive sector, along with lawmakers who were sounding the alarm.
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:49: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:88: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:30: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:152468 for open-end generation.
The attention mask is not set and cannot be inferred from input because pad token is same as eos token. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:104: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\infer_cli.py:232: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:30: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:152468 for open-end generation.
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:104: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\infer_cli.py:232: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:30: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
The attention mask and the pad token id were not set. As a consequence, you may observe unexpected behavior. Please pass your input's `attention_mask` to obtain reliable results.
Setting `pad_token_id` to `eos_token_id`:152468 for open-end generation.
D:\Repo_AI\bytedanceMegaTTS3\tts\frontend_function.py:104: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\infer_cli.py:232: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
with torch.cuda.amp.autocast(dtype=self.precision, enabled=True):
D:\Repo_AI\bytedanceMegaTTS3\tts\utils\audio_utils\io.py:35: RuntimeWarning: invalid value encountered in cast
wavfile.write(bytes_io, sr, wav.astype(np.int16))
| Saving results to ./gen/[P]As his long promised.wav
~~~
Contributor guide
No contributing guide indexed for this repository
Research direction
Reproduce the command in tts/infer_cli.py with both referenced prompt WAV files and compare the generated audio with the full input text. Then inspect tts/frontend_function.py and tts/utils/audio_utils/io.py, including the logged generation and saving paths. Done means the output preserves the complete sentence for both prompt voices without the reported truncation.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, pytorch
- Domain
- audio-video-rtc, machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100