OpenImagingLab / OpenImagingLab/FlashVSR

RuntimeError: FlashAttention only supports Ampere GPUs or newer.

Open
#2 8 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
1.9k
Forks
152
PR merge metrics
No merged PRs in 30d

Description

I followed your instructions. compiled Block-Sparse Attention, and with RTX 5090 cannot start code. Do you have solution?
python infer_flashvsr_tiny.py
C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\cuda_init_.py:63: FutureWarning: The pynvml package is deprecated. Please install nvidia-ml-py instead. If you did not install pynvml directly, please report this to the maintainers of the package that installed pynvml for you.
import pynvml # type: ignore[import]
0 NVIDIA GeForce RTX 5090
Loading models from: ./FlashVSR/diffusion_pytorch_model_streaming_dmd.safetensors
model_name: wan_video_dit model_class: WanModel
This model is initialized with extra kwargs: {'has_image_input': False, 'patch_size': [1, 2, 2], 'in_dim': 16, 'dim': 1536, 'ffn_dim': 8960, 'freq_dim': 256, 'text_dim': 4096, 'out_dim': 16, 'num_heads': 12, 'num_layers': 30, 'eps': 1e-06}
The following models are loaded: ['wan_video_dit'].

███████╗██╗ █████╗ ███████╗██╗ ██╗██╗ ██╗███████╗█████╗
██╔════╝██║ ██╔══██╗██╔════╝██║ ██║██║ ██║██╔════╝██╔══██╗
█████╗ ██║ ███████║███████╗███████║╚██╗ ██╔╝███████╗███████║
██╔══╝ ██║ ██╔══██║╚════██║██╔══██║ ╚████╔╝ ╚════██║██╔═██║
██║ ███████╗██║ ██║███████║██║ ██║ ╚██╔╝ ███████║██║ ██║
╚═╝ ╚══════╝╚═╝ ╚═╝╚══════╝╚═╝ ╚═╝ ╚═╝ ╚══════╝╚═╝ ╚═╝
⚡FlashVSR

Using wan_video_dit from ./FlashVSR/diffusion_pytorch_model_streaming_dmd.safetensors.
No wan_video_vae models available.

[example0.mp4] Original Resolution: 176x144 | Original Frames: 138 | FPS: 15
[example0.mp4] Scaled (x4.00): 704x576 -> Target (128-multiple): 640x512
[example0.mp4] Target Frames (8n-3): 133
C:\Users\admin\FlashVSR\examples\WanVSR\infer_flashvsr_tiny.py:38: UserWarning: The given NumPy array is not writable, and PyTorch does not support non-writable tensors. This means writing to this tensor will result in undefined behavior. You may want to copy the array to protect its data or make it writable before converting it to a tensor. This type of warning will be suppressed for the rest of this program. (Triggered internally at C:\actions-runner_work\pytorch\pytorch\pytorch\torch\csrc\utils\tensor_numpy.cpp:212.)
t = torch.from_numpy(np.asarray(img, np.uint8)).to(device=device, dtype=torch.float32) # HWC
0%| | 0/15 [00:00<?, ?it/s]
Traceback (most recent call last):
File "C:\Users\admin\FlashVSR\examples\WanVSR\infer_flashvsr_tiny.py", line 233, in
main()
File "C:\Users\admin\FlashVSR\examples\WanVSR\infer_flashvsr_tiny.py", line 219, in main
video = pipe(
^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\utils_contextlib.py", line 120, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\FlashVSR\diffsynth\pipelines\flashvsr_tiny.py", line 389, in call
noise_pred_posi, pre_cache_k, pre_cache_v = model_fn_wan_video(
^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\FlashVSR\diffsynth\pipelines\flashvsr_tiny.py", line 553, in model_fn_wan_video
x, last_pre_cache_k, last_pre_cache_v = block(
^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\FlashVSR\diffsynth\models\wan_video_dit.py", line 461, in forward
self_attn_output, self_attn_cache_k, self_attn_cache_v = self.self_attn(
^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\FlashVSR\diffsynth\models\wan_video_dit.py", line 362, in forward
x = self.attn(reorder_q, reorder_k, reorder_v, attention_mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\nn\modules\module.py", line 1775, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\nn\modules\module.py", line 1786, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\FlashVSR\diffsynth\models\wan_video_dit.py", line 296, in forward
x = flash_attention(q=q, k=k, v=v, num_heads=self.num_heads, attention_mask=attention_mask)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\FlashVSR\diffsynth\models\wan_video_dit.py", line 189, in flash_attention
x = block_sparse_attn_func(
^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\block_sparse_attn\block_sparse_attn_interface.py", line 399, in block_sparse_attn_func
return func.apply(
^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\torch\autograd\function.py", line 581, in apply
return super().apply(*args, **kwargs) # type: ignore[misc]
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\block_sparse_attn\block_sparse_attn_interface.py", line 192, in forward
out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = _block_sparse_attn_forward(
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Users\admin\anaconda3\envs\flashvsr\Lib\site-packages\block_sparse_attn\block_sparse_attn_interface.py", line 98, in _block_sparse_attn_forward
out, q, k, v, out_padded, softmax_lse, S_dmask, rng_state = block_sparse_attn_cuda.fwd_block(
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: FlashAttention only supports Ampere GPUs or newer.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce with python infer_flashvsr_tiny.py and start at diffsynth/models/wan_video_dit.py, especially flash_attention and the failing block_sparse_attn call. Check the installed block-sparse attention dependency and the RTX 5090 architecture path. Done should mean the supported behavior or required compatibility change is established and documented.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, pytorch
Domain
machine-learning, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
28/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.