babysor / babysor/MockingBird

使用mandarin_200k.pt模型报错

Open
#886 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
36.9k
Forks
5.2k
PR merge metrics
No merged PRs in 30d

Description

**Summary[问题简述(一句话)]**
使用mandarin_200k.pt模型报错,使用pretrained-11-7-21_75k.pt正常,由于安装依赖monotonic-align==0.0.3报错过不了,删除了==0.0.3成功安装

**Env & To Reproduce[复现与环境]**
环境:wsl2-ubuntu22.04,python3.9.0和python3.10都试过
代码版本:2023 年 3 月 7 日版本
模型:mandarin_200k.pt

**Screenshots[截图(如有)]**
![image](https://user-images.githubusercontent.com/21698755/233583526-5727f839-4555-430e-a3d1-cbd13dcfbbdc.png)
```
RuntimeError: Error(s) in loading state_dict for Tacotron: size mismatch for encoder.embedding.weight: copying a param with shape torch.Size([70, 512]) from checkpoint, the shape in current model is torch.Size([75, 512]). size mismatch for encoder_proj.weight: copying a param with shape torch.Size([128, 512]) from checkpoint, the shape in current model is torch.Size([128, 1024]). size mismatch for decoder.attn_rnn.weight_ih: copying a param with shape torch.Size([384, 768]) from checkpoint, the shape in current model is torch.Size([384, 1280]). size mismatch for decoder.rnn_input.weight: copying a param with shape torch.Size([1024, 640]) from checkpoint, the shape in current model is torch.Size([1024, 1152]). size mismatch for decoder.stop_proj.weight: copying a param with shape torch.Size([1, 1536]) from checkpoint, the shape in current model is torch.Size([1, 2048]).
```
**使用v0.0.1版本并使用mandarin_200k.pt模型同样报错
```
Synthesizer using device: cpu
using synthesizer model: synthesizer/saved_models/mandarin_200k.pt
Trainable Parameters: 31.951M
[2023-04-21 16:19:44,685] ERROR in app: Exception on /api/synthesize [POST]
Traceback (most recent call last):
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask_restx/api.py", line 674, in error_router
return original_handler(e)
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/home/liuxixigua/MockingBird/web/__init__.py", line 108, in synthesize
specs = current_synt.synthesize_spectrograms(texts, embeds)
File "/home/liuxixigua/MockingBird/synthesizer/inference.py", line 87, in synthesize_spectrograms
self.load()
File "/home/liuxixigua/MockingBird/synthesizer/inference.py", line 65, in load
self._model.load(self.model_fpath)
File "/home/liuxixigua/MockingBird/synthesizer/models/tacotron.py", line 523, in load
self.load_state_dict(checkpoint["model_state"], strict=False)
File "/home/liuxixigua/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Tacotron:
size mismatch for encoder.embedding.weight: copying a param with shape torch.Size([70, 512]) from checkpoint, the shape in current model is torch.Size([75, 512]).
127.0.0.1 - - [2023-04-21 16:19:44] "POST /api/synthesize HTTP/1.1" 500 401 3.199693
```

**使用v0.0.1版本并使用mandarin_200k.pt模型同样报错
```
Synthesizer using device: cpu
using synthesizer model: synthesizer/saved_models/pretrained-11-7-21_75k.pt
Trainable Parameters: 31.951M
[2023-04-21 16:24:38,117] ERROR in app: Exception on /api/synthesize [POST]
Traceback (most recent call last):
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 2528, in wsgi_app
response = self.full_dispatch_request()
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 1825, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask_restx/api.py", line 674, in error_router
return original_handler(e)
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 1823, in full_dispatch_request
rv = self.dispatch_request()
File "/home/liuxixigua/.local/lib/python3.9/site-packages/flask/app.py", line 1799, in dispatch_request
return self.ensure_sync(self.view_functions[rule.endpoint])(**view_args)
File "/home/liuxixigua/MockingBird/web/__init__.py", line 108, in synthesize
specs = current_synt.synthesize_spectrograms(texts, embeds)
File "/home/liuxixigua/MockingBird/synthesizer/inference.py", line 87, in synthesize_spectrograms
self.load()
File "/home/liuxixigua/MockingBird/synthesizer/inference.py", line 65, in load
self._model.load(self.model_fpath)
File "/home/liuxixigua/MockingBird/synthesizer/models/tacotron.py", line 523, in load
self.load_state_dict(checkpoint["model_state"], strict=False)
File "/home/liuxixigua/.local/lib/python3.9/site-packages/torch/nn/modules/module.py", line 2041, in load_state_dict
raise RuntimeError('Error(s) in loading state_dict for {}:\n\t{}'.format(
RuntimeError: Error(s) in loading state_dict for Tacotron:
size mismatch for encoder_proj.weight: copying a param with shape torch.Size([128, 1024]) from checkpoint, the shape in current model is torch.Size([128, 512]).
size mismatch for gst.stl.attention.W_query.weight: copying a param with shape torch.Size([512, 512]) from checkpoint, the shape in current model is torch.Size([512, 256]).
size mismatch for decoder.attn_rnn.weight_ih: copying a param with shape torch.Size([384, 1280]) from checkpoint, the shape in current model is torch.Size([384, 768]).
size mismatch for decoder.rnn_input.weight: copying a param with shape torch.Size([1024, 1152]) from checkpoint, the shape in current model is torch.Size([1024, 640]).
size mismatch for decoder.stop_proj.weight: copying a param with shape torch.Size([1, 2048]) from checkpoint, the shape in current model is torch.Size([1, 1536]).
127.0.0.1 - - [2023-04-21 16:24:38] "POST /api/synthesize HTTP/1.1" 500 401 3.134052
```

Contributor guide

No contributing guide indexed for this repository

Research direction

Start in web/__init__.py at the /api/synthesize entry point, then follow synthesizer/inference.py into synthesizer/models/tacotron.py and its load_state_dict call. Compare the model architecture and checkpoint dimensions for mandarin_200k.pt and pretrained-11-7-21_75k.pt. Done means mandarin_200k.pt loads without size-mismatch errors and the synthesis endpoint returns successfully.

Written by the indexing model from the issue text.

Assessment

Tech stack
flask, python, pytorch
Domain
audio-video-rtc, backend, machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.