modelscope / modelscope/DiffSynth-Studio
Wan2.2 animate inference errrors
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 13.1k
- Forks
- 1.3k
- Avg merge
- 13h 12m
- Merged PRs (30d)
- 45
Description
Hi, when i using diffsynth to do inference of wan2.2 animate, if the input video shape is not 1280(h)*720(w) will get the error as blow:
File "./DiffSynth-Studio/diffsynth/models/wan_video_animate_adapter.py", line 643, in after_patch_embedding
[rank3]: x[:, :, 1:] += pose_latents
[rank3]: RuntimeError: The size of tensor a (37) must match the size of tensor b (36) at non-singleton dimension 2
same error happened when i change the frames of input pose vedio(error when frames==147)
input_image = Image.open("./src_pose_1280_720.mp4").raw_data()[:147-4]
animate_face_video = VideoData("./src_face.mp4").raw_data()[:147-4]
But it seems work with https://github.com/Wan-Video/Wan2.2/blob/main/generate.py
Is there any solution? THX!
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at diffsynth/models/wan_video_animate_adapter.py:643 and reproduce the tensor mismatch with non-1280x720 input and 147 pose-video frames. Compare the corresponding inference path in Wan2.2's generate.py. Done means the animate inference path handles these inputs without the reported dimension error.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100