OpenImagingLab / OpenImagingLab/FlashVSR

infer_flashvsr_v1.1_tiny_long_video某个视频推理到一半报错

Open
#45 4 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
152
PR merge metrics
No merged PRs in 30d

Description

用infer_flashvsr_v1.1_tiny_long_video.py推理长视频,测试了十几个视频,结果都正常,
但有一个视频,固定推理到某步报错,应该怎么解决?

56%|██████████████████████████████████████████████████████████████▏ | 510/918 [07:45<06:12, 1.09it/s]
Traceback (most recent call last):
File "/home/liyuanqi/project/FlashVSR/infer_flashvsr_v1.1_tiny_long_video.py", line 287, in
main()
File "/home/liyuanqi/project/FlashVSR/infer_flashvsr_v1.1_tiny_long_video.py", line 257, in main
video = pipe(
^^^^^
File "/home/liyuanqi/miniconda3/envs/flashvsr/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/liyuanqi/project/FlashVSR/diffsynth/pipelines/flashvsr_tiny_long.py", line 394, in call
noise_pred_posi, pre_cache_k, pre_cache_v = model_fn_wan_video(
^^^^^^^^^^^^^^^^^^^
File "/home/liyuanqi/project/FlashVSR/diffsynth/pipelines/flashvsr_tiny_long.py", line 564, in model_fn_wan_video
x, last_pre_cache_k, last_pre_cache_v = block(
^^^^^^
File "/home/liyuanqi/miniconda3/envs/flashvsr/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/liyuanqi/miniconda3/envs/flashvsr/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/liyuanqi/project/FlashVSR/diffsynth/models/wan_video_dit.py", line 464, in forward
self_attn_output, self_attn_cache_k, self_attn_cache_v = self.self_attn(
^^^^^^^^^^^^^^^
File "/home/liyuanqi/miniconda3/envs/flashvsr/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1739, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/liyuanqi/miniconda3/envs/flashvsr/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1750, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/liyuanqi/project/FlashVSR/diffsynth/models/wan_video_dit.py", line 330, in forward
q = rope_apply(q, freqs, self.num_heads)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/liyuanqi/project/FlashVSR/diffsynth/models/wan_video_dit.py", line 269, in rope_apply
x_out = torch.view_as_real(x_out * freqs).flatten(2)
~~~~~~^~~~~~~
RuntimeError: The size of tensor a (64) must match the size of tensor b (42) at non-singleton dimension 3

望解答,谢谢!

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the failure with the problematic video using infer_flashvsr_v1.1_tiny_long_video.py. Trace the tensor shapes from diffsynth/pipelines/flashvsr_tiny_long.py into rope_apply in diffsynth/models/wan_video_dit.py, where the 64-versus-42 mismatch occurs. Done means the video completes successfully without breaking the other tested videos.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
computer-vision
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.