modelscope / modelscope/DiffSynth-Studio

Run Wan2.1-VACE-14B failed when set vace_video_mask

Open
#643 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

Run Wan2.1-VACE-14B failed when have vace_video_mask. error log:

Downloading Model from https://www.modelscope.cn to directory: DiffSynth-Studio-main/models/Wan-AI/Wan2.1-T2V-1.3B
2025-06-27 09:18:01,016 - modelscope - INFO - Target directory already exists, skipping creation.
VAE encoding: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:08<00:00,  1.05it/s]
VAE encoding: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 9/9 [00:08<00:00,  1.11it/s]
  0%|                                                                                                                                                                                 | 0/50 [00:00<?, ?it/s]
Traceback (most recent call last):
  File "DiffSynth-Studio-main/examples/wanvideo/model_inference/Wan2.1-VACE-14B.py", line 61, in <module>
    video = pipe(
        prompt=prompt,
    ...<6 lines>...
        seed=1, tiled=True
    )
  File "conda/envs/wan2.1/lib/python3.13/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "DiffSynth-Studio-main/diffsynth/pipelines/wan_video_new.py", line 556, in __call__
    noise_pred_posi = self.model_fn(**models, **inputs_shared, **inputs_posi, timestep=timestep)
  File "DiffSynth-Studio-main/diffsynth/pipelines/wan_video_new.py", line 1128, in model_fn_wan_video
    vace_hints = vace(x, vace_context, context, t_mod, freqs)
  File "conda/envs/wan2.1/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
           ~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^
  File "conda/envs/wan2.1/lib/python3.13/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
    return forward_call(*args, **kwargs)
  File "DiffSynth-Studio-main/diffsynth/models/wan_video_vace.py", line 61, in forward
    torch.cat([u, u.new_zeros(1, x.shape[1] - u.size(1), u.size(2))],
                  ~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: Trying to create tensor with negative dimension -53040: [1, -53040, 5120]

Same error raised after I changed the key of self.inputs_shared in class WanVideoPipeline from vace_video_mask to vace_mask, cause I found that vace_mask instead vace_video_mask is in the input_params of WanVideoUnit_VACE.

Help, please.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/wanvideo/model_inference/Wan2.1-VACE-14B.py and reproduce the failure using the reported vace_video_mask input. Then inspect WanVideoPipeline in diffsynth/pipelines/wan_video_new.py and the forward method in diffsynth/models/wan_video_vace.py, including the vace_mask input declared by WanVideoUnit_VACE. Done means the example no longer raises the negative-dimension error when a video mask is supplied.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.