modelscope / modelscope/DiffSynth-Studio

Wan2.1-VACE-14B Inference Bug

Open
#741 6 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
13.1k
Forks
1.3k
Avg merge
13h 12m
Merged PRs (30d)
45

Description

Thank you for your excellent work, but when running Wan2.1-VACE-14B inference, I encountered the following error when setting vace_video, vace_video_mask, and vace_reference_image simultaneously.

Traceback (most recent call last):
  File "/DiffSynth-Studio-New/examples/wanvideo/model_inference/Wan2.1-VACE-14B.py", line 42, in <module>
    video = pipe(
  File "/mnt/bn/maliva-gen-ai-v2/liwei.947/software/anaconda/envs/wanx/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
    return func(*args, **kwargs)
  File "/DiffSynth-Studio/diffsynth/pipelines/wan_video_new.py", line 449, in __call__
    noise_pred_posi = self.model_fn(**models, **inputs_shared, **inputs_posi, timestep=timestep)
  File "/DiffSynth-Studio/diffsynth/pipelines/wan_video_new.py", line 978, in model_fn_wan_video
    vace_hints = vace(x, vace_context, context, t_mod, freqs)
  File "/mnt/bn/maliva-gen-ai-v2/liwei.947/software/anaconda/envs/wanx/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1751, in _wrapped_call_impl
    return self._call_impl(*args, **kwargs)
  File "/mnt/bn/maliva-gen-ai-v2/liwei.947/software/anaconda/envs/wanx/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1762, in _call_impl
    return forward_call(*args, **kwargs)
  File "/DiffSynth-Studio/diffsynth/models/wan_video_vace.py", line 60, in forward
    c = torch.cat([
  File "/DiffSynth-Studio/diffsynth/models/wan_video_vace.py", line 61, in <listcomp>
    torch.cat([u, u.new_zeros(1, x.shape[1] - u.size(1), u.size(2))],
RuntimeError: Trying to create tensor with negative dimension -7200: [1, -7200, 5120]

I've noticed others experiencing the same error in Issues 1 and Issues 2. The resolution of my three inputs is consistent, and the total frame count of both videos is the same, so it seems unrelated to the inputs. I also tried the official VACE code, which worked fine.

I'm looking forward to your response and thank you for your help.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with examples/wanvideo/model_inference/Wan2.1-VACE-14B.py and diffsynth/models/wan_video_vace.py, then trace the call through diffsynth/pipelines/wan_video_new.py:978 using the three simultaneous inputs. Reproduce the negative-dimension failure and compare input-shape handling with the official VACE implementation; done means Wan2.1-VACE-14B inference succeeds for that combination without the runtime error.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.