modelscope / modelscope/DiffSynth-Studio

训练的时候报错 the size of tensor a (4) must match the size of tensor b (5) at non-singleton dimension 2

Open
#1,077 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

num_frames % 4 != 1. We round it up to 17.
num_frames % 4 != 1. We round it up to 17.
num_frames % 4 != 1. We round it up to 17.
num_frames % 4 != 1. We round it up to 17.
num_frames % 4 != 1. We round it up to 17.
num_frames % 4 != 1. We round it up to 17.
0%| | 0/1 [00:11<?, ?it/s]
[rank3]: Traceback (most recent call last):
[rank3]: File "/workspace/Wan2.1-main/Finetune/DiffSynth-Studio-main/train.py", line 136, in
[rank3]: launch_training_task(dataset, model, model_logger, args=args)
[rank3]: File "/workspace/Wan2.1-main/Finetune/DiffSynth-Studio-main/diffsynth/trainers/utils.py", line 559, in launch_training_task
[rank3]: loss = model(data)
[rank3]: ^^^^^^^^^^^
[rank3]: File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
[rank3]: return self._call_impl(*args, **kwargs)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl
[rank3]: return forward_call(*args, **kwargs)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/opt/conda/lib/python3.11/site-packages/deepspeed/utils/nvtx.py", line 20, in wrapped_fn
[rank3]: ret_val = func(*args, **kwargs)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/opt/conda/lib/python3.11/site-packages/deepspeed/runtime/engine.py", line 2179, in forward
[rank3]: loss = self.module(*inputs, **kwargs)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl
[rank3]: return self._call_impl(*args, **kwargs)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/opt/conda/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1603, in _call_impl
[rank3]: result = forward_call(*args, **kwargs)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/workspace/Wan2.1-main/Finetune/DiffSynth-Studio-main/train.py", line 90, in forward
[rank3]: loss = self.pipe.training_loss(**models, **inputs)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/workspace/Wan2.1-main/Finetune/DiffSynth-Studio-main/diffsynth/pipelines/wan_video_new.py", line 119, in training_loss
[rank3]: inputs["latents"] = self.scheduler.add_noise(inputs["input_latents"], inputs["noise"], timestep)
[rank3]: ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[rank3]: File "/workspace/Wan2.1-main/Finetune/DiffSynth-Studio-main/diffsynth/schedulers/flow_match.py", line 99, in add_noise
[rank3]: sample = (1 - sigma) * original_samples + sigma * noise
[rank3]: ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~
[rank3]: RuntimeError: The size of tensor a (4) must match the size of tensor b (5) at non-singleton dimension 2

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in train.py and follow the training_loss call into diffsynth/pipelines/wan_video_new.py, then inspect diffsynth/schedulers/flow_match.py where input_latents and noise are combined. Reproduce the training run and compare their shapes at add_noise; done means the tensors can be combined without the reported dimension mismatch and training proceeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.