babysor / babysor/MockingBird

运行web.py,更换synthesizer模型报错

Open
#891 0 comments 4 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
36.9k
Forks
5.2k
PR merge metrics
No merged PRs in 30d

Description

只有 pretrained_75k.pt 正确,选其它模型(如:my_run8_25k.pt 、qh3_53k.pt 、rty4_87k.pt 等),都报 torch.Size 错误。信息如下:

RuntimeError: Error(s) in loading state_dict for Tacotron: size mismatch for gst.stl.attention.W_query.weight: copying a param with shape torch.Size([512, 256]) from checkpoint, the shape in current model is torch.Size([512, 512]).
Traceback:
File "/home/armstrong/miniconda3/envs/torch2/lib/python3.11/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)
File "/tmp/tmp69jcq9tj.py", line 13, in
render_streamlit_ui()
File "/mnt/f/AI/digitalhuman/app/voices/mockingbird/src/control/mkgui/base/ui/streamlit_ui.py", line 909, in render_streamlit_ui
session_state.output_data = opyrator(input=input_data_obj)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/f/AI/digitalhuman/app/voices/mockingbird/src/control/mkgui/base/core.py", line 203, in __call__
return self.function(input_obj, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/f/AI/digitalhuman/app/voices/mockingbird/src/control/mkgui/app.py", line 140, in synthesize
specs = current_synt.synthesize_spectrograms(texts, embeds)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/mnt/f/AI/digitalhuman/app/voices/mockingbird/src/models/synthesizer/inference.py", line 91, in synthesize_spectrograms
self.load()
File "/mnt/f/AI/digitalhuman/app/voices/mockingbird/src/models/synthesizer/inference.py", line 69, in load
self._model.load(self.model_fpath, self.device)
File "/mnt/f/AI/digitalhuman/app/voices/mockingbird/src/models/synthesizer/models/base.py", line 55, in load
self.load_state_dict(state, strict=False)
File "/home/armstrong/miniconda3/envs/torch2/lib/python3.11/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with models/synthesizer/inference.py and models/synthesizer/models/base.py, following the load path shown in the traceback. Compare the checkpoint dimensions for pretrained_75k.pt and the failing synthesizer models, especially gst.stl.attention.W_query.weight. Done means supported model files load without the reported size mismatch and synthesis works for them.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
audio-video-rtc, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.